Variable-illumination fourier ptychographic imaging devices, systems, and methods

ABSTRACT

Certain aspects pertain to Fourier ptychographic imaging systems, devices, and methods such as, for example, high NA Fourier ptychographic imaging systems and reflective-mode NA Fourier ptychographic imaging systems.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/209,604, titled “VARIABLE-ILLUMINATION FOURIER PTYCHOGRAPHIC IMAGINGDEVICES, SYSTEMS, AND METHODS” and filed on Jul. 13, 2016, which is acontinuation of U.S. patent application Ser. No. 14/466,481 (issued asU.S. Pat. No. 9,497,379), titled “VARIABLE-ILLUMINATION FOURIERPTYCHOGRAPHIC IMAGING DEVICES, SYSTEMS, AND METHODS” and filed on Aug.22, 2014, which claims benefit of and priority to U.S. ProvisionalPatent Application No. 61/899,715, titled “Increasing Numerical Apertureof Dry Objective to Unity via Fourier Ptychographic Microscopy” andfiled on Nov. 4, 2013, U.S. Provisional Patent Application No.61/868,967, titled “Alternative Optical Implementations for FourierPtychographic Microscopy” and filed on Aug. 22, 2013, and U.S.Provisional Patent Application No. 62/000,722, titled “Ultra-High NAMicroscope via Fourier Ptychographic Microscopy” and filed on May 20,2014; all of these applications are hereby incorporated by reference intheir entireties and for all purposes.

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with government support under Grant No. OD007307awarded by the National Institutes of Health. The government has certainrights in the invention.

BACKGROUND

Certain embodiments described herein generally relate to imagingtechniques. More specifically, certain aspects pertain tovariable-illumination Fourier ptychographic imaging systems, devices,and methods that can be used in high resolution imaging applicationssuch as, for example, pathology, haematology, semiconductor waferinspection, and X-ray and electron imaging.

Imaging lenses ranging from microscope objectives to satellite-basedcameras are physically limited in the total number of features they canresolve. These limitations are a function of the point-spread function(PSF) size of the imaging system and the inherent aberrations across itsimage plane field of view (FOV). Referred to as the space-bandwidthproduct, the physical limitation scales with the dimensions of the lensbut is usually on the order of 10 megapixels regardless of themagnification factor or numerical aperture (NA). A discussion ofspace-bandwidth product of conventional imaging systems can be found inLohmann, A. W., Dorsch, R. G., Mendlovic, D., Zalevsky, Z. & Ferreira,C., “Space-bandwidth product of optical signals and systems,” J. Opt.Soc. Am. A. 13, pages 470-473 (1996), which is hereby incorporated byreference for this discussion. While conventional imaging systems may beable to resolve up to 10 megapixels, there is typically a tradeoffbetween PSF and FOV. For example, certain conventional microscopeobjectives can offer a sharp PSF (e.g., 0.5 μm) across a narrow FOV(e.g., 1 mm), while others imaging systems with wide-angle lenses canoffer a wide FOV (e.g., 10 mm) at the expense of a blurry PSF (e.g., 5μm).

Certain interferometric synthetic aperture techniques that try toincrease spatial-bandwidth product are described in Di, J. et al., “Highresolution digital holographic microscopy with a wide field of viewbased on a synthetic aperture technique and use of linear CCDscanning,”Appl. Opt. 47, pp. 5654-5659 (2008); Hillman, T. R., Gutzler,T., Alexandrov, S. A., and Sampson, D. D., “High-resolution, wide-fieldobject reconstruction with synthetic aperture Fourier holographicoptical microscopy,” Opt. Express 17, pp. 7873-7892 (2009); Granero, L.,Micó, V., Zalevsky, Z., and García, J., “Synthetic aperturesuperresolved microscopy in digital lensless Fourier holography by timeand angular multiplexing of the object information,”Appl. Opt. 49, pp.845-857 (2010); Kim, M. et al., “High-speed synthetic aperturemicroscopy for live cell imaging,” Opt. Lett. 36, pp. 148-150 (2011);Turpin, T., Gesell, L., Lapides, J., and Price, C., “Theory of thesynthetic aperture microscope,” pp. 230-240; Schwarz, C. J., Kuznetsova,Y., and Brueck, S., “Imaging interferometric microscopy,” Optics letters28, pp. 1424-1426 (2003); Feng, P., Wen, X., and Lu, R.,“Long-working-distance synthetic aperture Fresnel off-axis digitalholography,” Optics Express 17, pp. 5473-5480 (2009); Mico, V.,Zalevsky, Z., García-Martínez, P., and García, J., “Synthetic aperturesuperresolution with multiple off-axis holograms,” JOSA A 23,pp.3162-3170 (2006); Yuan, C., Zhai, H., and Liu, H., “Angularmultiplexing in pulsed digital holography for aperture synthesis,”Optics Letters 33, pp. 2356-2358 (2008); Mico, V., Zalevsky, Z., andGarcia, J., “Synthetic aperture microscopy using off-axis illuminationand polarization coding,” Optics Communications, pp. 276, 209-217(2007); Alexandrov, S., and Sampson, D., “Spatial informationtransmission beyond a system's diffraction limit using optical spectralencoding of the spatial frequency,” Journal of Optics A: Pure andApplied Optics 10, 025304 (2008); Tippie, A. E., Kumar, A., and Fienup,J. R., “High-resolution synthetic-aperture digital holography withdigital phase and pupil correction,” Opt. Express 19, pp. 12027-12038(2011); Gutzler, T., Hillman, T. R., Alexandrov, S. A., and Sampson, D.D., “Coherent aperture-synthesis, wide-field, high-resolutionholographic microscopy of biological tissue,” Opt. Lett. 35, pp.1136-1138 (2010); and Alexandrov, S. A., Hillman, T. R., Gutzler, T.,and Sampson, D. D., “Synthetic aperture Fourier holographic opticalmicroscopy,” Phil. Trans. R. Soc. Lond. A 339, pp. 521-553 (1992), allof which are hereby incorporated by reference for the discussion ofattempts to increase spatial bandwidth. Most of the above-describedinterferometric synthetic aperture techniques include setups that recordboth intensity and phase information using interferometric holographysuch as off-line holography and phase-shifting holography.Interferometric holography has its limitations. For example,interferometric holography recordings typically use highly coherentlight sources. As such, the constructed images typically suffer fromcoherent noise sources such as speckle noise, fixed pattern noise(induced by diffraction from dust particles and other opticalimperfections in the beam path), and multiple interferences betweendifferent optical interfaces. Thus the image quality is typically worsethan from a conventional microscope. On the other hand, using off-axisholography sacrifices spatial-bandwidth product (i.e., reduces totalpixel number) of the image sensor. A discussion of certain off-axisholography methods can be found in Schnars, U. and Jüptner, W. P. O.,“Digital recording and numerical reconstruction of holograms,”Measurement Science and Technology, 13, R85 (2002), which is herebyincorporated by reference for this discussion. In addition,interferometric imaging techniques may subject to uncontrollable phasefluctuations between different measurements. Hence, accurate a prioriknowledge of the sample location may be needed to set a reference pointin the image recovery process. Another limitation is that many of theseinterferometric imaging systems require mechanical scanning to rotatethe sample and thus precise optical alignments, mechanical control at asub-micron level, and associated maintenances are required by thesesystems. In terms of spatial-bandwidth product, these interferometricimaging systems may present little to no advantage as compared with aconventional microscope.

Previous lensless microscopy such as in-line holography andcontact-imaging microscopy also present drawbacks. For example,conventional in-line holography does not work well with contiguoussamples and contact-imaging microscopy requires a sample to be in closeproximity to the sensor. A discussion of certain digital in-lineholography devices can be found in Denis, L., Lorenz, D., Thiebaut, E.,Fournier, C. and Trede, D., “Inline hologram reconstruction withsparsity constraints,” Opt. Lett. 34, pp. 3475-3477 (2009); Xu, W.,Jericho, M., Meinertzhagen, I., and Kreuzer, H., “Digital in-lineholography for biological applications,” Proc. Natl Acad. Sci. USA 98,pp. 11301-11305 (2001); and Greenbaum, A. et al., “Increasedspace-bandwidth product in pixel super-resolved lensfree on-chipmicroscopy,” Sci. Rep. 3, page 1717 (2013), which are herebyincorporated by reference for this discussion. A discussion of certaincontact-imaging microscopy can be found in Zheng, G., Lee, S. A.,Antebi, Y., Elowitz, M. B. and Yang, C., “The ePetri dish, an on-chipcell imaging platform based on subpixel perspective sweeping microscopy(SPSM),” Proc. Natl Acad. Sci. USA 108, pp. 16889-16894 (2011); andZheng, G., Lee, S. A., Yang, S. & Yang, C., “Sub-pixel resolvingoptofluidic microscope for on-chip cell imaging,” Lab Chip 10, pages3125-3129 (2010), which are hereby incorporated by reference for thisdiscussion.

A high spatial-bandwidth product is very desirable in microscopy forbiomedical applications such as pathology, haematology, phytotomy,immunohistochemistry, and neuroanatomy. For example, there is a strongneed in biomedicine and neuroscience to image large numbers of histologyslides for evaluation. This need has prompted the development ofsophisticated mechanical scanning and lensless microscopy systems. Thesesystems increase spatial-bandwidth product using complex mechanisms withhigh precision to control actuation, optical alignment, and motiontracking. These complex mechanisms tend to be expensive to fabricate anddifficult to use and maintain.

BRIEF SUMMARY

Certain embodiments described herein generally relate to imagingtechniques. More specifically, certain aspects pertain tovariable-illumination Fourier ptychographic imaging systems, devices,and methods that can be used in high resolution imaging applicationssuch as, for example, pathology, haematology, semiconductor waferinspection, and X-ray and electron imaging.

Certain embodiments are directed to an ultra-high NA Fourierptychographic imaging system comprising a variable illuminator, anoptical system, and a radiation detector. The variable illuminator isconfigured to illuminate a sample at a plurality of incidence angles atdifferent times. The optical system comprises a lens with a high NA, thelens configured to filter light issuing from the sample. In one example,the high NA may be about 0.50, and in another example, the high NA maybe in the range of about 0.40 to about 0.50, etc. The plurality ofincidence angles and the high NA correspond to overlapping regions inthe Fourier domain that cover an expanded NA of greater than 1.0. In oneexample, adjacent overlapping regions in the plurality of overlappingregions may have an overlapping area of at least about 20% to 90% of thearea of one of the overlapping regions. In another example, the adjacentoverlapping regions may have an overlapping area of at least about 70%of the area of one of the overlapping regions. In another example, theadjacent overlapping regions may have an overlapping area of at leastabout 75% of the area of one of the overlapping regions. In anotherexample, the adjacent overlapping regions may have an overlapping areaof at least about 2% and 99.5% of the area of one of the overlappingregions. The radiation detector is configured to acquire a plurality ofintensity images, each intensity image corresponding to a differentincidence angle of the plurality of incidence angles. In some aspects,the ultra-high NA Fourier ptychographic imaging system further comprisesa processor configured to generate an image with a higher resolutionthan a resolution of the intensity images by iteratively updating theoverlapping regions in the Fourier domain with intensity imagemeasurements. In one aspect, the lens may be configured to filter lightfrom the sample by passing light received within its acceptance angle.In one aspect, the lens may be configured to filter light from thesample by passing light received within its acceptance angle. In oneaspect, the optical system comprises a collection optical elementconfigured to receive light reflected from the sample and the variableilluminator and the collection optical element are located to the sameside of the sample in an epi-illumination mode. In one aspect, the lensis configured to receive light reflected from the sample and thevariable illuminator and the lens optical element are located to thesame side of the sample in an epi-illumination mode.

In certain aspects, the ultra-high NA Fourier ptychographic imagingsystem of embodiments described herein may further comprise a variableilluminator comprising one or more circular rings of light elements. Inone aspect, each outer ring may have a larger number of light elementsthan an adjacent smaller diameter ring. In one aspect, each concentricring has at least 6 light elements. In one aspect, each concentric ringlight elements separated by at least about 30 degrees. In one aspect,each concentric ring has a diameter of more than about 20mm. In oneaspect, each concentric ring has a diameter of more than about 40mm.

Certain embodiments are directed to a reflective-mode Fourierptychographic imaging system comprising a variable illuminator, anoptical system and a radiation detector. The variable illuminatorconfigured to illuminate a sample at a plurality of incidence angles atdifferent times in an epi-illumination mode. The optical systemcomprises a filtering optical element having a filtering function. Theoptical system is configured to receive light reflected from the sampleand filter the light reflected from the sample using the filteringoptical element, wherein the plurality of incidence angles and thefiltering function correspond to overlapping regions in the Fourierdomain. The radiation detector is configured to acquire a plurality ofintensity images, each intensity image corresponding to a differentincidence angle of the plurality of incidence angles. In one aspect, thereflective-mode Fourier ptychographic imaging system further comprises aprocessor configured to generate an image with a higher resolution thana resolution of the intensity images by iteratively updating theoverlapping regions in the Fourier domain with intensity imagemeasurements. In one aspect, the filtering optical element is a lensconfigured to filter light by passing light received within itsacceptance angle. In one aspect, the variable illuminator comprises afirst set of circular rings of light elements centered about a centralaxis of the filtering optical element. In one aspect, the optical systemfurther comprises a beam splitter placed at a 45 degree angle behind thefiltering optical element and the filtering optical element isconfigured to filter light issued from the sample, the beam splitter isconfigured to receive light filtered by the filtering optical elementand passes half the filtered light to the radiation detector. In oneaspect, the optical system further comprises a secondary lens. In thiscase, the secondary lens is configured to receive illumination at aplurality of incidence angles from the variable illuminator and passesthe illumination to the beam splitter and the beam splitter isconfigured to pass half the illumination to the sample through thefiltering optical element.

In certain aspects, the reflective-mode Fourier ptychographic imagingsystem comprises a variable illuminator comprising a first set ofcircular rings of light elements centered about a central axis of thefiltering optical element. In one aspect, the optical system furthercomprises a beam splitter placed at a 45 degree angle and behind thefiltering optical element, and configured to pass about half theincident light and reflect about half the incident light, and thevariable illuminator further comprises a second set of circular rings oflight elements located to provide illumination reflected by the beamsplitter and through the filtering optical element to the sample. Inanother aspect, the optical system further comprises a beam splitterplaced at a 45 degree angle and behind the filtering optical element,and configured to pass about half the incident light and reflect abouthalf the incident light, and the variable illuminator further comprisesa second set of circular rings of light elements located to provideillumination reflected by the beam splitter and through the filteringoptical element to the sample.

These and other features are described in more detail below withreference to the associated drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of components of a variable-illuminationFourier ptychographic imaging system.

FIG. 2 depicts a schematic diagram of a side view of components of avariable-illumination Fourier ptychographic imaging device intrans-illumination mode.

FIG. 3A depicts an orthogonal view of components of an ultra-high NAvariable-illumination Fourier ptychographic imaging device with acircular variable illuminator.

FIGS. 3B and 3C depict an expansion in the Fourier domain for theultra-high NA configuration shown in FIG. 3A.

FIGS. 3D and 3E depict an expansion in the Fourier domain for anultra-high NA configuration shown in FIG. 3A modified with a circularvariable illuminator having two concentric rings.

FIG. 4 depicts an orthogonal view of components of an ultra-high NAvariable-illumination Fourier ptychographic imaging device with arectangular array variable illuminator.

FIG. 5 depicts an orthogonal view of components of avariable-illumination Fourier ptychographic imaging device inepi-illumination mode.

FIG. 6 depicts an orthogonal view of components of avariable-illumination Fourier ptychographic imaging device inepi-illumination mode.

FIGS. 7A and 7B depict an orthogonal view of components of avariable-illumination Fourier ptychographic imaging device inepi-illumination mode.

FIG. 8 is a flowchart of a variable-illumination Fourier ptychographicimaging method.

FIG. 9 is a flowchart of an example of certain sub-steps of one of thesteps of the method of FIG. 8.

FIG. 10A is a flowchart of an example of certain sub-steps of one of thesteps of the method of FIG. 8.

FIGS. 10B and 10C are schematic illustrations depicting components of avariable-illumination Fourier ptychographic imaging device intrans-illumination mode.

FIG. 10D is an illustration of certain steps of thevariable-illumination Fourier ptychographic imaging method describedwith reference to FIGS. 8 and 10A.

FIG. 11 is a flowchart of a variable-illumination Fourier ptychographicimaging method with tile imaging.

FIG. 12 is a block diagram of subsystems that may be present in avariable-illumination Fourier ptychographic imaging system.

DETAILED DESCRIPTION

Certain embodiments described herein pertain to variable-illuminationFourier ptychographic imaging systems, devices, and methods.

I. Variable-Illumination Fourier Ptychographic Imaging Systems

In certain aspects, a variable-illumination Fourier ptychographicimaging system comprises a variable illuminator, an optical system, anda radiation detector. In some cases, the system may be in communicationwith a processor or further comprise a processor (e.g., microprocessor).The variable illuminator can illuminate (e.g., with plane waveillumination) a sample being imaged from a plurality of incidenceangles. The optical system can receive light issuing from the sample andpropagate it to the radiation detector. The optical system comprises atleast one filtering optical element that can “filter” light typicallybased on its acceptance angle. The radiation detector receives filteredlight from the optical system, and measures the light intensitydistribution to capture a plurality of intensity images of the samplecorresponding to different incidence angles. Each intensity image isassociated with a region in Fourier space. In the case of a filteringoptical element in the form of a lens, the diameter of the regioncorresponds to the NA of the lens and the center of the regioncorresponds to the incidence angle of the illumination at that sampletime. The components of the Fourier ptychographic imaging system (e.g.,variable illuminator and filtering optical element) are configured toacquire intensity images in the spatial domain that correspond tooverlapping circular regions in the Fourier space to overlap by acertain amount and/or to cover a larger region (e.g., covering higherfrequencies). For example, the NA of the filtering optical element andthe number and locations of discrete light elements of a variableilluminator may be designed so that circular pupil regions in Fourierspace overlap by a certain amount. In one case, these components may bedesigned so that the circular regions associated with adjacent incidentangles overlap by a certain percentage (e.g., by about 70%, by about80%, by about 90%, etc.) in the Fourier domain. The overlapping imagedata in Fourier space can be iteratively stitched together to generate ahigher resolution image of the sample. In some cases, the variableillumination Fourier ptychographic imaging system can also correct foraberrations in the system including, for example, refocusing thehigher-resolution image.

In certain aspects, a variable-illumination Fourier ptychographicimaging system comprises an optical system with a low NA filteringoptical element (e.g., 2× lens with 0.08) for a wide field-of-view(e.g., 13 mm in diameter) of the sample. This system acquires intensityimages with relatively low resolution due to the low NA optical elementfiltering light issuing from the sample. These intensity imagescorrespond to smaller circular regions in Fourier space than if a higherNA optical element were used. In order to overlap these smaller circularregions in Fourier space by a certain amount (e.g., 70%, 75%, etc.), thevariable illuminator in this system is configured to provideillumination with relatively short spacing (e.g., 0.05 rad) betweenadjacent incidence angles. Examples of variable-illumination Fourierptychographic systems with low NA filtering optical element for widefield-of-view imaging can be found in U.S. patent application Ser. No.14/065,280, titled “Fourier Ptychographic Imaging Systems, Devices, andMethods” and filed on Oct. 28, 2013 and in U.S. patent application Ser.No. 14/065,305, titled “Fourier Ptychographic X-ray Imaging Systems,Devices, and Methods,” and in G. Zheng, R. Horstmeyer and C. Yang,“Wide-field, high-resolution Fourier ptychographic microscopy,” NaturePhotonics, 2013, which are both hereby incorporated by reference intheir entirety for details of these systems.

In other aspects, an ultra-high NA (e.g., NA greater than 1.0)variable-illumination Fourier ptychographic imaging system is configuredto achieve finer resolution of a sample image. In these aspects, theultra-high NA variable-illumination Fourier ptychographic imaging systemcomprises an optical system with a higher NA filtering optical element(e.g., 20× lens with 0.5 NA) and a higher illumination NA for a combinedincreased system NA. The higher NA filtering optical element allowsthese systems to capture higher resolution intensity images than withthe low NA system described above. These intensity images correspond tolarger regions in Fourier space than intensity images captured with alower NA filtering optical element. Since larger regions are covered,the variable illuminator can be configured with reduced spacing betweenadjacent incidence angles and with a reduced number N of incidenceangles. In these systems, fewer intensity images may be needed togenerate the same or higher resolution than with systems using a low NAfiltering optical element. Since fewer intensity images may be needed,the image acquisition time is shorter and may require fewer resources togenerate an image with the same or higher resolution than the low NAsystem. Also, the variable illuminator can be of a simpler design (e.g.,less dense LED matrix) since fewer light elements are needed to provideillumination from the reduced number N of incidence angles. In somecases, the variable illuminator may be further configured so that thedifference between extreme incidence angles is larger (i.e., higherillumination NA) than with the low NA system described above. That is, ahigher illumination NA allows for capturing of high frequency data atthe outer regions in Fourier space which also improves the resolution ofthe final images. Thus, these variable-illumination Fourierptychographic imaging systems with an increased illumination NA and/oran increased optical system NA can provide for an increased system NAthat can improve resolution of the images. That is, these systems may beable to illuminate the sample with incidence angles that allow foracquisition of images that cover larger overlapping regions in Fourierspace and higher frequency data. When combined, these overlapping largerregions can result in a synthesized large system NA region that may, incertain cases, be close to unity. In certain cases, these systems have ahigh synthetized system NA (e.g., close to unity where the intrinsic NAof the filtering light element is lower such as, for example, about0.75) while maintaining a large working distance, and without usingneeding an immersion medium.

In conventional microscopes, the highest system NA that can be achievedis limited by geometric principle (i.e. at most the entire upperhemisphere light cone of light from the sample is collected) and lensdesign technology, resulting in an upper bound of ˜0.95 for drymicroscope and ˜1.40 for oil immersion microscope. Some conventionalwater or oil immersion objectives may provide NA>0.9 where an immersionmedia with refractive index greater than 1 improves collection of lightfrom the sample. However, immersion objectives have several drawbacksthat may make them unsuitable for some applications. Firstly, samplesneed to be immersed in media and typically the working distance is veryshort (0.1-0.2 mm), which presents an obstacle for micromanipulation ofthe sample. Secondly, common immersion media have inherently highabsorption characteristics in the ultraviolet region (<375 nm) and nearinfrared region (>700 nm) of the spectrum, which brings some problem tothe bright-field immersion microscopy in this region and alsofluorescence immersion microscopy. A description of the relationshipbetween oil immersion and numerical aperture can be found at:http://www.olympusmicro.com/primer/anatomy/immersion.html, which ishereby incorporated by reference for this description.

In certain cases, variable-illumination Fourier ptychographic imagingsystems described herein may be configured to operate in atrans-illumination mode (i.e. directing illumination source through thesample and toward collection optical element) and/or in anepi-illumination mode (i.e., directing illumination source toward sampleand away from collection optical element). In the epi-illumination mode,the collection optical elements received reflected light from thesurface of the sample. In order to operate in the epi-illumination mode,the illumination source (e.g., illuminated element of the variableilluminator) may be configured to direct illumination to the sample fromthe same side as where the collection optical element is located. Someexamples of variable-illumination Fourier ptychographic imaging devicesshown operating in the epi-illumination mode are shown in FIGS. 5, 6,and 7A-7B. In trans-illumination mode, reflected light may not becaptured by the collection optical element and it may be that only lighttransmitted through the sample is collected. Thus, an epi-illuminationmode may be more effective for imaging thick and/or non-transparentsamples than a trans-illumination mode. The variable-illuminationFourier ptychographic imaging systems operating in epi-illumination modetypically image reflective surfaces of the sample. Configuringvariable-illumination Fourier ptychographic imaging systems forepi-illumination mode may be particularly useful in applications thatinvolve metal or semiconductor surface inspection including, forexample, semiconductor wafer, chip, and/or electronic circuit boardinspection, among others. Some applications for these Fourierptychographic imaging systems configured for epi-illumination mode mayinclude hand-held cameras with a modified flash system or satelliteimagery.

FIG. 1 is a block diagram of components of a variable-illuminationFourier ptychographic imaging system 10, according to certainembodiments. The variable-illumination Fourier ptychographic imagingsystem 10 comprises a variable-illumination Fourier ptychographicimaging device 100 and an optional (denoted by dashed line) computingdevice 200 in electronic communication with variable-illuminationFourier ptychographic imaging device 100. In certain illustratedexamples, such as the one shown in FIG. 1, a sample is shown provided tothe variable-illumination Fourier ptychographic imaging device for animage measurement process. It will be understood that the sample in notan essential component of the device, and is being shown for thepurposes of illustrating an operation of the device. The optionalcomputing device 200 can be in various forms such as, for example, asmartphone, laptop, desktop, tablet, etc. Various forms of computingdevices would be contemplated by one skilled in the art.

The variable-illumination Fourier ptychographic imaging device 100comprises a variable illuminator 110, an optical system 130, and aradiation detector 140. The variable illuminator 110 is configured toprovide illumination at a plurality of N incidence angles at (θ_(i,j),θy_(i,j)), i=1 to n, j=1 to m to the sample 20. The variable illuminator110 is configured to illuminate the sample 20 in a trans-illuminationmode and/or in an epi-illumination mode. In the trans-illumination mode,the variable illuminator 110 directs illumination through the sample 20and toward a collection optical element of the optical system 130. In anepi-illumination mode, the variable illuminator 110 directs illuminationto the sample 20 and away from a collection optical element of theoptical system 130.

The optical system 130 comprises components configured to receive lightissuing from the sample 20 and propagate it to the radiation detector140. A collection optical element of the optical system 130 receiveslight issued from the specimen 20. Either the collection optical elementor another optical element of the optical system 130 filters the lightit receives. For example, this filtering optical element may be in theform of an objective lens, which accepts light within its acceptanceangle to act as a filter. The optical system 130 propagates the filteredlight to the radiation detector 140, which measures (e.g., records) anintensity distribution at the radiation detector 140 at M sample times,t_(q=1 to M), to capture a plurality of M intensity images of thesample. In certain cases, M=N, i.e. an intensity measurement correspondsto each incidence angle.

In FIG. 1, the optional computing device 200 comprises a processor 210(e.g., a microprocessor), a computer readable medium (CRM) 220 incommunication with the processor 210, and a display 230 also incommunication with the processor 210. The processor 210 is in electroniccommunication with the radiation detector 140 to receive signal(s) withimage data corresponding to M intensity images. The image data mayinclude, for example, intensity distributions, associated acquisitiontimes, etc. The intensity images are of the sample 20 and/or the areaaround the sample 20.

The processor 210 is in electronic communication with CRM 220 (e.g.,memory) to be able to transmit signals with image data in order to storeto and retrieve image data from the CRM 220. Processor 210 is inelectronic communication with display 230 to be able to send image dataand instructions to display images and other output, for example, to auser of the system 10. As shown by a dotted line, the variableilluminator 110 may optionally be in electronic communication withprocessor 210 to send instructions for controlling variable illuminator110. For example, in certain aspects these control instructions may beimplemented to synchronize the illumination times at different incidenceangles with the sample times of the radiation detector 140. Theelectronic communication between components of system 10 and othersystems and devices described herein may be in wired or wireless form.

The processor 210 may also receive instructions stored on the CRM 220and execute those instructions to perform one or more functions ofvariable-illumination Fourier ptychographic imaging system 10. Forexample, the processor 210 may execute instructions to perform one ormore steps of the variable-illumination Fourier ptychographic imagingmethod. As another example, the processor 210 may execute instructionsfor illuminating light elements of the variable illuminator 110. Asanother example, the processor 210 may execute instructions stored onthe CRM 220 to perform one or more other functions of the system suchas, for example, 1) interpreting image data from the plurality ofintensity images, 2) generating a higher resolution image from the imagedata, and 3) displaying one or more images or other output from thevariable-illumination Fourier ptychographic imaging method on thedisplay 230.

The CRM (e.g., memory) 220 can store instructions for performing certainfunctions of the system 10. These instructions are executable by theprocessor 220 or other processing components of the system 10. The CRM220 can also store the (lower resolution) intensity and higherresolution image data, and other data produced by the system 10.

The variable-illumination Fourier ptychographic imaging system 10 alsoincludes a display 230 in electronic communication with the processor210 to receive data (e.g., image data) and provide display data to thedisplay 230 for, for example, an operator of the variable-illuminationFourier ptychographic imaging system 10. The display 230 may be a colordisplay or a black and white display. In addition, the display 230 maybe a two-dimensional display or a three-dimensional display. In oneembodiment, the display 230 may be capable of displaying multiple views.

In one operation, the variable-illumination Fourier ptychographicimaging system 10 performs a method comprising a measurement process, arecovery process, and an optional display process. During themeasurement process, the sample is illuminated from a plurality of Nincidence angles (θx_(i,j), i=1 to n, j=1 to m, (N=n×m) using thevariable illuminator 110. The optical system 130 has a filtering opticalelement that filters light issuing from the sample. The optical system130 propagates the filtered light to the radiation detector 140. Theradiation detector 140 receives the filtered light and acquires aplurality of M intensity images, I_(k,l), k=1 to o and j=1 to p, whereM=o×p. In certain cases, M may be N. The variable illuminator 110 isconfigured to generate illumination at incidence angles that willgenerate image data in Fourier space that overlaps by a certain amount.During the recovery process, the M intensity images are iterativelycombined in Fourier space to generate a higher-resolution image data(intensity and/or phase). During the optional display process, an image(e.g., higher-resolution image, acquired intensity image, etc.) and/orother output may be provided on a display 230. In certain aspects, thesystem 10 may also be able to correct for any aberrations in the system10, including re-focusing of the higher-resolution image. In one case,the system 10 may also be able to propagate the higher resolution imageto one or more planes. The image data from these propagated images atdifferent planes can be used to generate a three-dimensional image. Incertain aspects, the system 10 may also be able to generate an image atdifferent illumination wavelengths (RGB) to generate a color image.

Certain modifications, additions, or omissions may be made to thevariable-illumination Fourier ptychographic imaging system 10 withoutdeparting from the scope of the disclosure. In addition, the componentsof the variable-illumination Fourier ptychographic imaging system 10 orthe components of the variable-illumination Fourier ptychographicimaging devices described herein may be integrated or separatedaccording to particular needs. For example, the computing device 200 orcomponents thereof may be integrated into the variable-illuminationFourier ptychographic imaging device 100. In some embodiments, theprocessor 210 or other suitable processor may be part of thevariable-illumination Fourier ptychographic imaging device 100. In somecases, the processor 210 may be integrated into a radiation detector sothat the radiation detector performs the functions of the processor 210.As another example, the CRM 220 and/or display 230 may be omitted fromthe variable-illumination Fourier ptychographic imaging system 100 incertain cases.

In certain aspects, the variable-illumination Fourier ptychographicimaging systems and devices may further comprise a receptacle forreceiving the sample at a sample surface. The sample surface may be partof a component of or a separate component of the systems and devices.

In certain aspects, one or more of the full field-of-view intensityimages captured by a variable-illumination Fourier ptychographic imagingsystem 10 may be divided into one or more tile images. In these cases,the processor may construct a higher resolution complex image for eachtile independently, and then combine the tile images to generate thefull field-of-view image. This ability to process tile imagesindependently allows for parallel computing. In these aspects, each tilemay be represented by a two-dimensional area. In polar spatialcoordinates, each tile may be a circular area or an oval area. Inrectilinear spatial coordinates, the full field-of view low resolutionimage may be divided up into a two-dimensional matrix of tiles in arectangular area. In some embodiments, the dimensions of atwo-dimensional square matrix of tiles may be in powers of two whenexpressed in number of pixels of the radiation detector such as, forexample, a 256 by 256 matrix, a 64×64 matrix, etc.

FIG. 2 depicts a schematic diagram of a side view of components of avariable-illumination Fourier ptychographic imaging device 100(a) intrans-illumination mode, according to certain embodiments. Thevariable-illumination Fourier ptychographic imaging device 100(a) is anexample of one configuration of the variable-illumination Fourierptychographic imaging device 100 described with respect to FIG. 1.

In FIG. 2, the variable-illumination Fourier ptychographic imagingdevice 100(a) comprises a variable illuminator 110, an optical system130, and a radiation detector 140 having a sensing surface 142. Thevariable illuminator 110 comprises a light element 112 and a surface111. The variable illuminator 110 also comprises an x′-axis, a y′-axis(not shown) at a plane depicting the approximated plane from which thesource of illumination is provided, and a z′-axis. Although FIG. 2 showsvariable illuminator 110 comprising a single light element 112, thevariable illuminator 110 may have additional light elements at differentlocations to provide incident light at a plurality of incidence angles.Although radiation detector 140 is shown at a distance away from opticalsystem 130, radiation detector 140 may optionally be located at theoptical system 130.

In the illustrated example, the sample 20 has been provided to aspecimen surface 126 for the measurement process. The light element 112is shown providing illumination 114 in a trans-illumination mode throughthe sample 20 where the illumination 114 has a wavevector kx_(i,j),ky_(i,j) for the measurement process. Also shown is an in-focus plane122 at z=0 and a sample plane 124 at z=z₀. The variable-illuminationFourier ptychographic imaging device 100(a) further comprises an x-axis,a y-axis (not shown) at the in-focus plane 122, and a z-axis orthogonalto the in-focus plane 122. Also shown is a distance d between thevariable illuminator 110 and the sample plane 124 and a working distanced_(o) between the sample 20 and the optical system 130. Generally, aworking distance, d₀, refers to the distance between the sample 20 andthe collecting optical element of the optical system 130.

In FIG. 2, the light element 112 is shown providing illumination 114 ata single sample (acquisition) time in the measurement process. Theoptical system 130 receives and filters light issuing from specimen 20.Light filtered by the optical system 130 is received at the sensingsurface 142 of the radiation detector 140. The radiation detector 140measures the intensity distribution of incident light received at thesensing surface 142 and captures an intensity image at the sample time.Although the variable-illumination Fourier ptychographic imaging device100(a) is shown at a single sample time, the device 100(a) may include Nlight elements 112 illuminating at, for example, N incidence angles(θx_(i,j), θy_(i,j)), i=1 to n, j=1 to m, where N=n×m. In this case, theradiation detector 140 may acquire a plurality of M intensity imagesI_(k,l), k=1 to o and j=1 to p at the M sample times, where eachintensity image may be acquired while the illumination is at a differentincidence angle of the plurality of N incidence angles (θx_(i,j),θy_(i,j)). The incidence angles (θx_(i,j), θy_(i,j)) are angles measuredrelative to an axis normal to the sample plane at z=z₀ and through pointP. In the side view shown in FIG. 2, only the component θx_(i,j) of theincidence angle in the x-z plane is shown.

A variable illuminator generally refers to a device that can beconfigured to provide incident radiation to the sample being imaged atdifferent incidence angles at M image acquisition times. In many cases,the variable illuminator is designed to provide incident radiation at aplurality of N incidence angles (θx_(i,d), θy_(i,j)), i=1 to n, j=1 tom. Generally, N has a value in a range from 2 to 1000. Each incidenceangle corresponds to a location of the corresponding acquired image datain Fourier space. Adjacent incidence angles in the spatial domaincorrespond to neighboring regions in Fourier space. In certain aspects,the variable illuminator is designed to provide illumination atincidence angles that provide for an overlapping area of neighboringregions of image data in Fourier space where the overlapping area is ofat least a certain minimum amount (e.g. 75% overlap, 70% overlap, 80%overlap, etc.). To provide this minimum amount of overlap of neighboringregions in Fourier space, the variable illuminator may be configured sothat the difference between adjacent incidence angles in the pluralityof N incidence angles is less than a certain maximum angular difference.That is, the variable illuminator may be configured with a maximumdifference between adjacent incidence angles to provide the minimumamount of overlap in Fourier space. For example, the maximum angulardifference may be about 0.05 rad for a 2×0.08 NA objective lens. Inanother case, the maximum angular difference may be about 0.1 rad.

In certain cases, the variable-illumination Fourier ptychographicimaging systems may include a filtering optical element in the form of alens having an acceptance angle. This acceptance angle corresponds tothe diameter of a circular pupil region in Fourier space. In thesecases, the variable illuminator may be configured to have adjacentincidence angles that are separated by an angle of a value defined bythe acceptance angle of the lens. In one case, the value of thedifference between two adjacent incidence angles of the plurality ofincidence angles may be in the range of about 10% to about 90% of theacceptance angle of the filtering optical element in the form of anobjective lens. In another case, the value of the difference between twoadjacent incidence angles of the plurality of incidence angles may be inthe range of 33% and 66% of the acceptance angle of the filteringoptical element in the form of an objective lens. In another case, thevalue of the difference between two adjacent incidence angles of theplurality of incidence angles may be less than about 76% of theacceptance angle of the filtering optical element in the form of anobjective lens. In another case, the difference between adjacentincidence angles is about ⅓ of the acceptance angle defined thefiltering optical element in the form of an objective lens. In anothercase, the range of incidence angles, defined by a difference between thelargest and smallest incidence angles, may be about equal to thenumerical aperture consistent with the spatial resolution of the finalhigher-resolution image. In one case, the acceptance angle is in therange of about −0.08 rad to about 0.08 rad, and the adjacent angle is0.05 rad.

The variable illuminator comprises one or more radiation sources.Although the radiation source(s) are usually coherent radiation sources,incoherent radiation source(s) may also be used in some cases andcomputational corrections may be applied. The radiation sources may bevisible light other forms of radiation. In cases that use visible lightradiation, the radiation source(s) is a visible light source. Someexamples of a radiation source of visible light include a liquid crystaldisplay (LCD) pixel and a pixel of a light emitting diode (LED) display.In cases that use other forms of radiation, other sources of radiationmay be used. For example, in embodiments that use X-ray radiation, theradiation source may comprise an X-ray tube and a metal target. Asanother example, in cases that use microwave radiation, the radiationsource may comprise a vacuum tube. As another example, in embodimentsthat use acoustic radiation, the radiation source may be an acousticactuator. As another example, in embodiments that use Terahertzradiation, the radiation source may be a Gunn diode. One skilled in theart would contemplate other sources of radiation. In one case that usesTerahertz radiation, the frequencies of the radiation provided by theillumination source may be in the range of about 0.3 to about 3 THz. Inone case that uses microwave radiation, the frequencies of the radiationprovided by the variable illuminator may be in the range of about 100MHz to about 300 GHz. In one case that uses X-ray radiation, thewavelengths of the radiation provided by the variable illuminator may bein the range of about 0.01 nm to about 10 nm. In one case that usesacoustic radiation, the frequencies of the radiation provided by thevariable illuminator may be in the range of about 10 Hz to about 100MHz.

In certain cases, the variable illuminator may comprise a plurality ofdiscrete light elements, each light element comprising at least oneradiation source. For example, a variable illuminator that is configuredto provide visible light typically includes a plurality of discretelight elements. Some examples of discrete light elements that canprovide visible light are an LCD pixel and a pixel of an LED display. Inmany cases, the illumination provided by each light element may beapproximated as plane wave illumination at the sample from a singleincidence angle. For example, the light element 112 in FIG. 2 providesillumination 114 at an incidence angle that has a component θx_(i,j) inthe x-z plane.

In certain cases, the properties (e.g., wavelength, frequency, phase,amplitude, polarity, etc.) of illumination from the activated radiationsource(s) of the variable illuminator at each acquisition time may beapproximately uniform. In some cases, the illumination from theactivated radiation source(s) at all acquisition times from allincidence angles may be approximately uniform. In other cases, theproperties may vary at the different incidence angles, for example, byproviding n different wavelengths λ₁, . . . , λ_(n) during themeasurement process. In other cases, the variable illuminator mayprovide RGB illumination of three wavelengths λ₁, λ₂, and λ₃corresponding to red, green, blue colors, respectively. In examples thatuse Terahertz radiation, the frequencies of the radiation provided bythe variable illuminator may be in the range of about 0.3 to about 3THz. In examples that use microwave radiation, the frequencies of theradiation provided by the variable illuminator may be in the range ofabout 100 MHz to about 300 GHz. In examples that use X-ray radiation,the wavelengths of the radiation provided by the variable illuminatormay be in the range of about 0.01 nm to about 10 nm. In examples thatuse acoustic radiation, the frequencies of the radiation provided by thevariable illuminator may be in the range of about 10 Hz to about 100MHz.

In some cases, the variable illuminator comprises a plurality of Nstationary discrete light elements at different spatial locations (e.g.,variable illuminator 110(a) in FIG. 3A). These N stationary lightelements may illuminate, individually or in groups of one or more, atdifferent sample times (e.g., successively) to provide illumination fromthe plurality of N incidence angles. In other cases, the variableilluminator may comprise a moving light element. This moving lightelement may move relative to the optical system, the sample, and theradiation detector, which may be kept stationary. In these cases, themoving light element may be moved to a plurality of N different spatiallocations using a mechanism such as a raster scanner. Based on therelative movement between the stationary components and the moving lightelement, the light element can provide illumination from the pluralityof N incidence angles. In other cases, the variable illuminatorcomprises a stationary light element and the other components of systemare moved to different spatial locations to provide the relativemovement. Based on this relative movement between the stationary lightelement and the other components of the system, the light element canprovide illumination from the plurality of N incidence angles.

In cases having a variable illuminator comprising a plurality of lightelements, the light elements may be in various arrangements such as aline grid, a rectangular grid, one or more concentric circles (rings), ahexagonal grid, curvilinear grid, or other suitable arrangement capableof providing the illumination from the plurality of incidence angles. Anexample of a circular variable illuminator 110(b) having light elementsin the form a single ring is shown in FIG. 3A. An example of rectangulararray variable illuminator 110(c) in the form of a rectilinear grid oflight elements is shown in FIG. 4. Some examples of light elements are apixel of a liquid crystal display (LCD) or a light emitting diode (LED).The arrangement of light elements may be configured with a spacingbetween adjacent elements and at particular locations that whenactivated can provide illumination at a plurality of incidence anglesthat correspond to overlapping regions in Fourier space, in some cases,with an overlap of a certain amount.

In cases with multiple light elements, the light elements locations maybe represented by a one-dimensional or two-dimensional array (e.g., 1×9array, 3×6 array, 10×10 array, 15×15 array, 32×32 array, 100×100 array,50×10 array, 20×60 array, or other array with two dimensions). In somecases, such a two-dimensional array has dimensions n×m with lightelement locations k_(i,j) (r, θ) or X_(i,j) (x, y), i=1 to n, j=1 to mwhere the number of locations, where N=n×m.

In certain aspects, the variable illuminator comprises discrete lightelements that are illuminated at different acquisition times in anorder, for example, according to illumination instructions. For example,the order may define the illumination times of individual light elementsor groups of light elements in a two-dimensional array of discrete lightelements. In one example where the two-dimensional matrix of lightelements is a rectangular array, a central light element may bedetermined. The illumination instructions may instruct to illuminate thecentral light element first, then illuminate the 8 light elementssurrounding the central light element going counterclockwise, thenilluminate the 16 light elements surrounding the previous light elementsgoing counterclockwise, and so on until the variable illuminator hasprovided illumination from the plurality of N incidence angles(θx_(i,j), θy_(i,j)), i=1 to N. In another example where thetwo-dimensional matrix of light elements is a polar matrix such as oneor more concentric rings, the illumination instructions may instruct toilluminate the light elements at smallest radius first (e.g., inclockwise, counterclockwise, or random order), then illuminate any lightelement at a larger radius, and so on until all the variable illuminatorhas provided illumination from the plurality of N incidence angles(θx_(i,j), θ_(i,j)), i=1 to N. In another example where thetwo-dimensional array of light elements is a rectangular or a polararray, a light element closest to the specimen may be determined. Theillumination instructions may instruct to illuminate the light elementclosest to the specimen, and then illuminate the light element nextclosest to the specimen, and then illuminate the light element nextclosest, and so on until the N light elements have been illuminated fromthe plurality of N incidence angles. In another example, the lightelements may be illuminated in a random order. In another example, asequential column by column order may be followed such as, for example,(X₁,Y₁), (X₁, Y₂), (X₁, Y₃), . . . (X₁, Y_(n)), (X₂, Y₁), (X₁, Y₂), (X₁,Y₃), . . . (X₂,Y_(n)), . . . (X_(m), Y_(n)). Alternatively, a row by roworder may be followed.

In certain aspects, a variable illuminator of certain systems describedherein may provide in an epi-illumination mode and/or in atrans-illumination mode. To be able to function in the epi-illuminationmode, the variable illuminator is typically located on the same side ofthe sample as the collecting optical element of the optical system. Tobe able to function in the trans-illumination mode, the variableilluminator is typically located on the opposite side of the sample asthe collecting optical element of the optical system.

A sample being imaged by the variable-illumination Fourier ptychographicimaging systems described herein can be comprised of one or more objectsand/or one or more portions of an object. Each object may be, forexample, a biological entity, an inorganic entity, etc. Some examples ofbiological entities that can be imaged include whole cells, cellcomponents, microorganisms such as bacteria or viruses, and cellcomponents such as proteins. An example of an inorganic entity that canbe imaged is a semiconductor wafer. In certain aspects, a thick and/ornon-transparent sample can be imaged by certain Fourier ptychographicimaging systems described herein. The sample may be provided in a mediumsuch as a liquid.

In luminescence imaging examples, a reagent (e.g.,fluorescence/phosphorescence dye) may be mixed with the sample to markor tag portions under investigation with fluorophore. A fluorophore canrefer to a component of a molecule that causes the molecule to fluoresceor phosphoresce. A fluorophore can absorb energy from excitation lightof a specific wavelength(s) and re-emit the energy at a differentwavelength(s). In luminescence imaging examples, the illumination sourcemay illuminate the sample with excitation light of predeterminedwavelength(s) (e.g., blue light) to activate the fluorophore in thesample. In response, the fluorophore release emissions of a differentwavelength(s) (e.g., red light).

The optical system 130 comprises one or more other components such as,for example, lens(es), beam splitter(s), objective(s), tube lens(es),wavelength filter(s), aperture element(s) (e.g., objective, physicaliris, etc.), and other like elements. In luminescence imaging example,the optical system 130 may include, for example, a filter (e.g.,material that passes emissions and blocks excitation light) between thecollection optics and the radiation detector to filter out excitationlight and pass emissions. The optical system 130 may include, forexample, certain microscope optical components or camera opticalcomponents. Generally, the optical system 130 comprises a collectionoptical element or first optical element that collects light issuingfrom the sample 20. The optical system 130 also comprises a filteringoptical element for filtering light issuing from the sample. Thefiltering optical element may be the collection optical element. Incertain cases, the filtering optical element may be a lens (e.g., anobjective lens). In certain ultra-high NA examples, the high NA of thelens may be about 0.50. In other ultra-high NA examples, the high NA ofthe lens may be in the range of about 0.50 to about 0.75. In anotherultra-high NA example, the high NA of the lens may be about 0.60.

In certain variable-illumination Fourier ptychographic imaging systemsdescribed herein, the radiation detector (e.g., radiation detector 140in FIG. 1) is configured to acquire a plurality of intensity images of asample by measuring/recording an intensity distribution of incidentradiation at a detector plane at a particular sample (acquisition) time.During an image measurement process, for example, the radiation detectormay acquire a plurality of M intensity images at M sample times,t_(q=1 to M). If visible light radiation is being measured, theradiation detector may be in the form of a charge coupled device (CCD),a CMOS imaging sensor, an avalanche photo-diode (APD) array, aphoto-diode (PD) array, a photomultiplier tube (PMT) array, or likedevice. If using THz radiation, the radiation detector may be, forexample, an imaging bolometer. If using microwave radiation, theradiation detector may be, for example, an antenna. If X-ray radiationis used, the radiation detector may be, for example, an x-ray sensitiveCCD. If acoustic radiation is used, the radiation detector may be, forexample, a piezoelectric transducer array. These examples of radiationdetectors and others are commercially available. In some cases, theradiation detector may be a color detector e.g. an RGB detector. Inother cases, the radiation detector need not be a color detector. Incertain cases, the radiation detector may be a monochromatic detector.

In certain aspects, a variable-illumination Fourier ptychographicimaging system comprises a variable illuminator configured to illuminatethe sample from a plurality of N illumination incidence angles andradiation detecrtor configured to capture a plurality of M intensityimages based on different incidence angles of the plurality of Nincidence angles. In certain cases, N=M (i.e. an intensity image isacquired for each illumination angle).

In certain aspects, the radiation detector may have discrete elements(e.g., pixels). The discrete detecting elements may be of any suitablesize (e.g., 1-10 microns) and any suitable shape (e.g., circular,rectangular, square, etc.). For example, a CMOS or CCD element may be1-10 microns and an APD or PMT light detecting element may be as largeas 1-4 mm. In one example, the radiation detecting element is a squarepixel having a size of 5.5 um.

A sample time or acquisition time can refer to a time that the radiationdetector 130 captures an intensity image of the sample. During certainimage measurement processes described here, the radiation detectorcaptures a plurality of M intensity images (e.g., M=1, 2, 5, 10, 20, 30,50, 100, 1000, 10000, etc.). At each sample time, t_(q) that anintensity image is captured, light is being provided to the sample at adifferent incidence angle of the plurality of N incidence angles. Incertain cases, the sampling rates may range from 0.1 to 1000 frames persecond.

Fourier space may refer to a mathematical space spanned by wave vectorskx and ky being the coordinate space in which the two-dimensionalFourier transforms of the spatial images created by theaperture-scanning Fourier ptychographic imaging system reside. Fourierspace may also refer to the mathematical space spanned by wavevectors kxand ky in which the two-dimensional Fourier transforms of the spatialimages collected by the radiation sensor reside.

During the measurement process, the radiation detector 130 capturesimage data comprising the plurality of M intensity images. The radiationdetector 130 may also generate other image data such as the sample timesand other related sample data. Each of the plurality of M intensityimages captured by the radiation detector is associated with a region inFourier space. In Fourier space, neighboring regions may share anoverlapping area over which they sample the same Fourier domain data.This overlapping area in Fourier space corresponds to the overlappingarea of neighboring incidence angles of the illumination provided by thevariable illuminator. In certain aspects, the variable illuminator isconfigured to provide illumination at a plurality of incidence anglesthat are spaced to provide a certain amount of overlapping area in theFourier domain data. In one case, the variable illuminator is configuredto provide illumination at a plurality of incidence angles to generatean overlapping area in the Fourier domain data in the range of about 2%to about 99.5% of the area of one of the regions. In another case, theoverlapping area between neighboring regions may have an area that is inthe range of about 65% to about 75% the area of one of the regions. Inanother case, the overlapping area between neighboring regions may havean area that is about 65% of the area of one of the regions. In anothercase, the overlapping area between neighboring regions may have an areathat is about 70% of the area of one of the regions. In another case,the overlapping area between neighboring regions may have an area thatis about 75% of the area of one of the regions.

Based on the geometry of the system 10, the variable illuminator may beconfigured to generate illumination from the incidence angles thatprovide a certain amount of overlap area between overlapping regions inFourier space. For example, the distance between light elements may beof a certain spacing (e.g., 1 mm, 0.5 mm, etc.). In FIG. 10B, thespacing between light elements is 4 mm.

Certain variable illumination Fourier ptychographic imaging systemsdescribed herein can be used for luminescence (e.g., fluorescence,phosphorescence, chemluminescence, bioluminescence, etc.) imaging. Forexample, certain systems may be adapted to collect emissions directedback toward the illumination source. In fluorescence imaging and otherluminescence imaging applications, fluorophores in the sample areexcited by excitation illumination of a certain wavelength(s) from theillumination source and emit light of a different wavelength(s)(emissions). These emissions tend to have a weak signal compared to theexcitation light so that collection efficiency may be important. Certainsystems may be configured to provide epi-illumination so that theradiation detector can receive emissions from the sample and/or lightreflected from the sample back toward the illumination source. Thesesystems have optical arrangements that can accommodate an illuminationsource that directs excitation illumination to the sample and away fromnext element in the system. In this way, propagation of the excitationillumination through the system may be substantially avoided.

Ultra-High NA Configurations

FIG. 3A depicts an illustration of an orthogonal view of components of avariable-illumination Fourier ptychographic imaging device 100(b),according to certain embodiments. The variable-illumination Fourierptychographic imaging device 100(b) is an example of an ultra-high NAconfiguration of the variable-illumination Fourier ptychographic imagingdevice of the system 10 described with respect to FIG. 1.

In FIG. 3A, the variable-illumination Fourier ptychographic imagingdevice 100(b) comprises a circular variable illuminator 110(b), anoptical system 130(b) having an objective 134 (e.g., microscopeobjective) and a tube lens 132, and a radiation detector 140(b). In thisillustration, the objective 134 is the collection (first) opticalelement of the optical system 130. The objective 132 has a relativelyhigh NA (e.g., in the range of about 0.60 to about 0.75). A sample 20 isshown on a specimen surface 126 as provided to the variable-illuminationFourier ptychographic imaging device 100(b).

In FIG. 3A, the variable-illumination Fourier ptychographic imagingdevice 100(b) comprises a circular variable illuminator 110(b) havingnine (9) discrete light elements 112(b) arranged in a single ring. Inother cases, the circular variable illuminator 110(b) may be in the formof a multiple concentric rings, or in other arrangements. In theillustrated example, the angular spacing between adjacent light elements112(b) is 40 degrees and the diameter of the ring is 40 mm. In othercases, the angular spacing between adjacent light elements (e.g., LEDs)may be about 2 degrees. In other cases, the angular spacing betweenadjacent light elements (e.g., LEDs) may be in a range of between about2 degrees to 40 degrees. In other cases, the diameter of the ring(s) maybe in the range of about 20 mm to 40 mm.

In certain aspects, a variable-illumination Fourier ptychographicimaging system may include a circular variable illuminator with lightelements arranged in one or more concentric rings (e.g. 1, 2, 3, etc.).In FIG. 3A, for example, the circular variable illuminator 110(b)comprises light elements in the form of a single ring. The diameters ofmulti-ring arrangements may be in the range of about 10 mm to about 60mm. In many cases, the light elements in each ring are equi-spaced(separated by a uniform angular difference between adjacent lightelements), however, other spacings may be used. In many cases, each ringwill have a different number of light elements. In other cases, eachring will have the same number of light elements.

Using a circular variable illuminator with light elements arranged inone or more concentric circles e.g., those with equi-spaced lightelements, can help improve uniformity of overlapping information. Thisuniformity may result in improved image quality as compared with imagesfrom systems that use variable illuminators with light elements in otherarrangements. For example, in cases where the rectangular array variableilluminator has a rectangular grid arrangement of elements, the expandedregion in Fourier space may not be as uniform in the radial direction.An example of an expanded region in Fourier domain from a rectangulargrid arrangement of light elements is shown in FIG. 10D. As you can seefrom the illustrations in FIGS. 3B and 3C associated with the systemusing light elements arranged in concentric rings, the expanded region280 in Fourier domain is substantially circular so that the informationin the higher frequencies associated with moving out radially will besubstantially uniform. In comparison, the expanded region in FIG. 10Dassociated with a rectangular arrangement of light elements issubstantially rectangular so that the information at the higherfrequencies will not be as uniform.

In FIG. 3A, each light element 112(b) is illustrated as an LED, althoughother types of light elements can be used. In this example, each lightelement 112(b) has a radiation source when illuminated. As denoted bythe dotted line, each light element 112(b) sequentially and individuallylights up to provide illumination 114 with a wavevector of (kx, ky). Inthis case, the sample 20 can be illuminated from 9 different incidenceangles by illumination provided by the each of the 9 light element112(b). In one example operation, the sample 20 is illuminated from 9different incidence angles at different acquisition times, the opticalsystem 130(b) collects light issuing from the illuminated sample 20, theobjective lens 134 filters light issuing from the sample based on itsacceptance angle, the tube lens focuses the filtered light to theradiation detector 140(b), and the radiation detector 140(b) capturesnine (9) intensity images at the acquisition times.

In FIG. 3A, the circular variable illuminator 110(b) is located toprovide illumination 114 in a trans-illumination mode i.e. illumination114 is directed through the sample 20. In another case, the variableilluminator 110(b) may be located to provide illumination in anepi-illumination mode, e.g., located on the same side of the sample 20as the objective lens 134.

In certain aspects, illumination from a variable illuminator at anincidence angle approximates plane wave illumination. Illumination by anoblique plane wave with a wavevector (kx, ky) is generally equivalent toshifting the center of the sample's spectrum by (kx, ky) in the Fourierdomain. Here, kx=k₀·cosx (cosine of angle between illuminationwavevector and x-axis); ky=k₀·cosy (cosine of angle between illuminationwavevector and

$k_{0} = {\frac{2\pi}{\lambda}.}$

The pupil function (i.e. coherent optical transfer function) of thefiltering optical element (e.g., objective lens 134 in FIG. 3A) inFourier space can be described as a circular shape low-pass filter witha radius of NA_(obj)·k₀ which is

${NA}*\frac{2\pi}{\lambda}$

in this case, where NA_(obj) is of the filtering optical element. Thus,each intensity image acquired by the radiation detector based on theapproximated plane wave illumination with wavevector (kx, ky) from thevariable illuminator contains sample's spectrum information centered atabout (kx, ky) in the Fourier domain. With illumination having awavevector of (kx ky) or (k₀·cosx, k₀·cosy), the image captured by thesystem contains spatial frequency information as high ask₀·[NA_(obj)+√{square root over ((cos²x+cos²y))}], where √{square rootover ((cos²x+cos²y))}=NA_(ill) is the numerical aperture of theillumination. The synthesized NA of the system can be described asNA_(syn)=NA_(obj)+NA_(ill).

To exceed unity NA_(sys) in a variable-illumination Fourierptychographic imaging system, components are configured such that theNA_(obj)+NA_(ill) sums up to greater than 1. For example, by using theultra-high NA configuration shown in FIG. 3A with a circular variableilluminator having a circular ring of 9 light elements (e.g., LEDs), theNA_(ill)=0.70 and with an filtering optical element in the form of anobjective lens having NA_(obj)=0.75 (e.g., 40×, 0.75 NA microscopeobjective lens), the resulting dry objective system has a NA_(syn)=1.45while retaining the field-of-view, and working distance of the objectivelens. As another example, by using the using the ultra-high NAconfiguration shown in FIG. 3A with an oil immersion setup having a100×1.4 NA objective for image acquisition and another 100×1.4NA forillumination (by imaging the light elements at the back focal plane ofthe objective which could form collimated illumination withNA_(ill)=1.4), the NA_(sys) could be as high as 2.8.

In some aspects, an iterative recovery process can be used to stitch theinformation at each of these regions associated with the plurality ofincidence angles to expand the information in the Fourier domain tocapture higher frequency information at the outer regions and to captureuniformly overlapping and wider regions of information, which can resultin higher resolution images of the sample. This expansion of theintrinsic NA_(obj) of the filtering optical element may generate anexpanded synthetic NA of the system.

In certain ultra-high NA variable-illumination Fourier ptychographicimaging systems described herein, the filtering optical element has arelatively high NA in order to capture higher frequency information foreach incidence angles, which corresponds to a wider circular region foreach incidence angle in the Fourier domain, which can result in an imagehaving a better resolution than about 400 nm. For example, avariable-illumination Fourier ptychographic imaging system with thevariable-illumination Fourier ptychographic imaging device 110(b) shownin FIG. 3A is a ultra-high NA configuration. In this example, theobjective lens 134 has a relatively high NA, for example, in a range ofabout 0.6 to about 0.75. In addition, the variable illuminator 110(b)has nine (9) light elements (e.g., LEDs) in s ring. FIG. 3B is anillustration depicting the expansion in the Fourier domain for thisultra-high NA configuration shown in FIG. 3A, of an embodiment. FIG. 3Cis a the illustration of FIG. 3B shown on a white background forclarification of certain details.

Certain variable-illumination Fourier ptychographic imaging systemsdescribed herein use angularly varying illumination to acquire highfrequency information about the sample. In certain cases, such as with asystem having the ultra-high NA configuration shown in FIG. 3A, thesystem acquires higher frequency information by using a higher NAfiltering optical element and/or by increasing the range of incidenceangles used by the variable illuminator. Using an iterative recoveryprocess (e.g. iterative phase retrieval process), the high frequencyinformation about the sample can be “stitched” together in the Fourierdomain, such as shown in FIGS. 3B-3C and 3D-3E, which means that anexpanded synthesized NA and finer resolution has been generated in thespace domain.

In FIGS. 3B and 3C, the center circular region 250 represents the rangeof information that can be captured by the objective lens 134 (e.g.,NA=0.60). Each of the nine (9) overlapping circular regions 260represents the range of information captured by the same objective lens134 at oblique angle illumination. Each overlapping circular region 260corresponds to one of the nine (9) different incidence angles. Thecircular region 280 shows the range of information captured by theobjective 134 at the (9) different incidence angles. For reference, acircular region 270 is illustrated to show the range of informationcaptured by a unity NA objective. As shown, the circular region 280 ofthe range of information captured by the objective 134 at the (9)different incidence angles is larger than the circle 270 of the unity NAobjective i.e. the NA of the configuration shown in FIG. 3B is greaterthan 1.0. That is, by overlapping circular regions in Fourier space, thecombined region can form an NA of more than 1.0. In configurations wherethe intrinsic NA of the objective 134 may be lower than 0.6, more LEDscan be arranged (either circularly or in a square array) to provideenough illumination angle, such that the area inside NA=1.0 can be fullyoccupied in the Fourier domain. An example of such a configuration isdescribed with reference to FIGS. 3D and 3E.

With oil immersion technology, a conventional microscope can achieve amaximum NA of 1.0. Using a variable-illumination Fourier ptychographicimaging system in a ultra-high NA configuration, such as with thevariable-illumination Fourier ptychographic imaging device 100(b) shownin FIG. 3A, the NA of the filtering optical element is relative high andthe resulting expanded NA of the system has been shown to exceed 1.0.

FIG. 3D is an illustration depicting the expansion in the Fourier domainfor an ultra-high NA configuration similar to the one shown in FIG. 3A,but with a variable illuminator 110(b) having two concentric circles(rings) of light elements (four elements on an inner ring and 12 lightelements on an outer ring) and with an objective having an NA of 0.50,according to an embodiment. The inner ring has four (4) light elementsand the outer ring has twelve (12) light elements. FIG. 3E is theillustration of FIG. 3D shown on a white background for clarification ofcertain details.

In FIGS. 3D and 3E, the center circular region 252 represents the rangeof information that can be captured by an objective lens having NA=0.50.The four (4) overlapping circular regions 262 (corresponding to theinner ring of the variable illuminator) represent the range ofinformation captured by the objective lens with NA=0.50 at oblique angleillumination at four corresponding incidence angles. Each overlappingcircular region 262 corresponds to one of the four (4) differentincidence angles. The twelve (12) overlapping circular regions 264(corresponding to the outer ring of the variable illuminator) representthe range of information captured by the objective lens with NA=0.50 atoblique angle illumination at 12 corresponding incidence angles. Eachoverlapping circular region 264 corresponds to one of twelve (12)different incidence angles.

The circular region 282 shows the expanded range of information capturedby the objective 134 having an NA of 0.50 at 16 different incidenceangles. For reference, a circular region 270 is illustrated to show therange of information captured by a unity NA objective. As shown, thecircular region 282 of the expanded range of information captured by theobjective at the sixteen (16) different incidence angles is larger thanthe circle 270 of the unity NA objective.

FIG. 4 depicts an illustration of an orthogonal view of components of avariable-illumination Fourier ptychographic imaging device 100(c),according to certain embodiments. The variable-illumination Fourierptychographic imaging device 100(c) is an example of an ultra-high NAconfiguration of the variable-illumination Fourier ptychographic imagingdevice of the system 10 described with respect to FIG. 1.

In FIG. 4, the variable-illumination Fourier ptychographic imagingdevice 100(c) comprises a rectangular array variable illuminator 110(c),an optical system 130(c) having an objective 134 (e.g., microscopeobjective) and a tube lens 132, and a radiation detector 140(c). In thisillustration, the objective 134 is the collection (first) opticalelement of the optical system 130. The objective 132 has a relativelyhigh NA (e.g., in the range of about 0.50 to about 0.75). A sample 20 isshown on a specimen surface 126 as provided to the variable-illuminationFourier ptychographic imaging device 100(c).

In FIG. 4, the rectangular array variable illuminator 110(c) is locatedto provide illumination 114 in a trans-illumination mode i.e.illumination 114 is directed through the sample 20. In another case, thevariable illuminator 110(c) may be located to provide illumination in anepi-illumination mode, e.g., located on the same side of the sample 20as the objective lens 134.

In FIG. 4, the variable-illumination Fourier ptychographic imagingdevice 100(c) comprises a variable illuminator 110(c) having lightelements 112(c) in a rectangular grid arrangement with 225 equi-spacedlight elements that corresponds to a 15×15 square array. Other numbersand arrangements of light elements can be used. In the illustratedexample, the spacing between adjacent light elements 112(c) is in arange of about 2 degrees to about 40 degrees.

In FIG. 4, each light element 112(c) is illustrated as an LED, althoughother types of light elements can be used. In this example, each lightelement 112(c) has a radiation source when illuminated. Duringoperation, each light element 112(c) sequentially and individuallylights up to provide illumination 114 with a wavevector of (kx, ky). Inthis case, the sample 20 can be illuminated from 225 different incidenceangles by illumination provided by the each of the 225 light element112(c). In one example operation, the sample 20 is illuminated from 225different incidence angles at 225 different acquisition times, theoptical system 130(c) collects light issuing from the illuminated sample20, the objective lens 134 filters light issuing from the sample basedon its acceptance angle, the tube lens focuses the filtered light to theradiation detector 140(c), and the radiation detector 140(c) captures225 intensity images at the 225 acquisition times.

FIGS. 5, 6, and 7A-7B depict schematic diagrams of side views ofcomponents of reflection-mode configurations (configurations inepi-illumination mode) of the variable-illumination Fourierptychographic imaging device of the system 10 described with respect toFIG. 1. Each of the variable-illumination Fourier ptychographic imagingdevices shown in FIGS. 5, 6, and 7A-7B is configured to locate avariable illuminator on the same plane (e.g., FIG. 5) or behind theplane (e.g. FIGS. 6 and 7A-B) of the imaging optics. These illustrateddevices are shown in epi-illumination mode. Some primary applicationsfor systems with such devices in epi-illumination mode include metal orsemiconductor surface inspection, including semiconductor wafer, chip,and/or electronic circuit board inspection, among others. Secondaryapplications may extend to include any scenario in which thevariable-illumination Fourier ptychographic imaging system 10 of FIG. 1can be applied with epi-illumination such as with hand-held cameras witha modified flash system, or satellite imagery. The examples shown inFIGS. 5, 6, and 7A-7B can include components configured for anultra-high NA system. For example, the NA of the objective lens in FIGS.7A-7B can have an NA of about 0.50.

FIG. 5 depicts an illustration of an orthogonal view of components of avariable-illumination Fourier ptychographic imaging device 100(d),according to certain embodiments. The variable-illumination Fourierptychographic imaging device 100(d) comprises a circular variableilluminator 110(d), an optical system 130(d) comprising a filteringoptical element in the form of an imaging lens 137, and a radiationdetector 140(d) having a detector plane 142. A sample 20 is shown on aspecimen surface 126 as provided to the variable-illumination Fourierptychographic imaging device 100(d).

In FIG. 5, the imaging lens 137 has a focal length f, a radius r, and anacceptance angle 2θ_(A). The imaging lens 137 may have an NA in therange of about 0.60 to about 0.75. In the illustrated example, theimaging lens 137 may be similar to a large camera lens so that theworking distance d_(o) is large such as, for example, about 10-20 cm. Inother examples, a smaller lens may be uses, such as a microscope lens,in which case the working distance d_(o) would be smaller such as, forexample, 2-3 cm.

In FIG. 5, the cicular array variable illuminator 110(d) comprises lightelements 112 (e.g., LEDs) arranged in 12 concentric rings (e.g.,circular LED rings) equally spaced between each ring and centered arounda central axis and around the imaging lens 137. Other numbers concentricrings may be used in other cases such as 1, 2, 3, 4, 5, 6, etc. In thisillustrated example, the light elements 112 are located at the sampleplane of the imaging lens 137. In other cases, the light elements 112may be at an offset plane, but remain on the same side of the sample 20as the imaging lens 137 in order to provide illumination in aepi-illumination mode. In the illustrated example, the rings areequi-spaced from each other with a radial spacing defined as Δr. In thisillustrated example, the variable-illumination Fourier ptychographicimaging device 100(d) has a variable illuminator 110(d) that is locatedat a distance, equal to the working distance d_(o), above the sample 20to provide epi-illumination mode.

In FIG. 5, the resolution variable-illumination Fourier ptychographicimaging device 100(d) is shown at a single illumination time and/oracquisition time. At this time, a single light element 112 of thevariable illuminator 110(d) is activated to provide illumination 114 atan incidence angle of θ_(B) with a wavevector of (kx,ky). At othertimes, the other light elements 112 may be providing illumination. In anexample operation of a system comprising the variable illuminator of thevariable-illumination Fourier ptychographic imaging device 100(d), thevariable illuminator 110(d) generates illumination 114 to the sample 20at a plurality of N incidence angles. The imaging lens 137 receiveslight from the sample 20 within its acceptance angle to filter thelight. The optical system 130 propagates the filtered light to theradiation detector 140(d), which measures an intensity distribution tocapture an intensity image at different incidence angles.

The illustrated example also includes a distance d_(i) between theimaging lens 137 and the radiation detector 140(d) and a workingdistance d₀ between the imaging lens 137 and the sample 20. In oneexample, the Fourier ptychographic imaging device 100(d) may have thefollowing relative dimensions: f=5 cm; d_(i) 7.02 cm; d_(o)=17.3 cm;r=0.25 cm; θ_(B)=30 degrees; and θ_(A)=3 degrees.

The variable-illumination Fourier ptychographic imaging device 100(d) ofFIG. 5 includes a variable illuminator 110(d) that does not have lightelements at the center where the the imaging lens 137 is located.Without light elements at the center, the images generated by the device110(d) with this illuminator 110(d) will not include low spatialfrequencies. In some applications, such as characterization ofslowly-varying phase objects, or when accurate knowledge of thereflectance from the entire object surface is required, this low spatialfrequency information may be valuable. The configuration shown in FIG. 5has a large working distance and a simple design with few components.Since the configuration does not collect information at low spatialfrequencies, this configuration is ideally suited for imaging of highresolution features or defects, for example, in chip inspectionapplications.

FIG. 6 depicts an illustration of an orthogonal view of components of avariable-illumination Fourier ptychographic imaging device 100(e),according to certain embodiments. Certain components of thevariable-illumination Fourier ptychographic imaging device 100(e) aresimilar to those of the variable-illumination Fourier ptychographicimaging device 100(d) shown in FIG. 5. In FIG. 6, thevariable-illumination Fourier ptychographic imaging device 100(e) isconfigured to capture low spatial frequencies that may be omitted by theconfiguration shown in FIG. 5. This variable-illumination Fourierptychographic imaging device 100(e) is configured to capture low spatialfrequencies by comprising a beamsplitter 139 and a second smaller set ofconcentric rings 110(e)(2) of light elements 112(2) on the other side ofthe imaging lens 138 (imaging optics) so that the light elements 112(2)are directed toward the image plane of the imaging optics. The secondset of light elements 112(2) are focused through the imaging optics toilluminate the sample with a plane wave at the sample plane. In certaincases, the configuration shown in FIG. 6 includes a larger aperture thanthe configuration shown in FIG. 5. The configuration shown in FIG. 6 mayprovide a large working distance.

In FIG. 6, the variable-illumination Fourier ptychographic imagingdevice 100(e) comprises a variable illuminator including a first set ofconcentric rings 110(e)(1) and a second set of concentric rings110(e)(2), an optical system including comprises an imaging lens 138 anda beam splitter 139, and a radiation detector 140(e) having a detectorplane 142. A sample 20 is shown on a specimen surface 126 as provided tothe variable-illumination Fourier ptychographic imaging device 100(e).The illustrated example shows a working distance d_(o) between theimaging lens 138 and the sample 20. The illustrated example alsoincludes a distance d_(i) between the imaging lens 138 and the radiationdetector 140(e).

The beam-splitter 139 is configured to transmit half the illuminationincident at a 45 degree angle to the beam-splitter 139 and not absorbedby the beam-splitter 139. The remaining half of the incidentillumination (not absorbed) is reflected by the beam-splitter 139. Forexample, the the beam splitter 139 may be comprised of a sheet of glassor other substrate with a coating designed to control the lightaccordingly. As another example, a beam splitter may be a half-silveredmirror with a continuous thin coating of reflective material (e.g.,metal). Another example is a swiss cheese beam splitter which has adiscontinuous coating with holes to obtain the desired ratio ofreflection to transmission.

The imaging lens 138 has a focal length f, a radius r, and an acceptanceangle of 2θ_(A). In the illustrated example, the imaging lens 138 isconfigured to filter light by accepting light within its acceptanceangle, 2θ_(A). An example of values that can be used in the illustratedconfiguration are: f=6 cm, r=1 cm, and θ_(A)=5 degrees. Other focallengths, radii, and acceptance angles can be used. To maintain a largelens-sample distance, the imaging lens 138 has a relatively low NA inthe range of about 0.1 to about 0.3. In the illustrated example, theimaging lens 138 has an NA of about 0.16, which is a relatively low NA(e.g., about 0.08, about 0.09, about 0.10, in a range of between about0.07 to about 0.20, etc.).

In the illustrated example, the imaging lens 138 may be, for example, alarge camera lens having a focal length f of 6 cm and a radius r of 2cm. If using a large camera lens, the variable-illumination Fourierptychographic imaging device 100(e) will have a corresponding largeworking distance d_(o) such as, for example, about 10-20 cm. In otherexamples, a smaller lens may be uses such as a microscope lens, in whichcase the working distance d_(o) would be smaller such as, for example,2-3 cm. In the illustrated example, d_(o)=12 cm and d_(i)=12 cm; othervalues may be used.

In FIG. 6, the optical path distance between the beam splitter 139 andthe second set of concentric rings 110(e)(2) is designated as b and theoptical path distance between the beam splitter 139 and the imaging lens138 is designated as a. In the illustrated example, the optical systemis configured so that the imaging lens 138 is located at a combinedoptical path distance of a+b=f from the second set of concentric rings110(e)(2).

In FIG. 6, the variable illuminator of the variable-illumination Fourierptychographic imaging device 100(e) comprises two sets of concentricrings (e.g., circular LED arrays) of light elements: a first set oftwelve (12) equally-spaced concentric rings 110(e)(1) (e.g., a first LEDarray) and a second set of eight (8) equally-spaced concentric rings110(e)(2) (e.g., a second LED array). Other numbers of concentric ringsmay be used in other cases such as 1, 2, 3, 4, 5, 6, etc. The first setof concentric rings 110(e)(1) comprises light elements 112(1) located atthe plane of the imaging lens 138 and centered around the imaging lens138. In other cases, the light elements 112(1) may be at one or moreoffset planes on the same side of the sample 20 as the imaging lens 138to be configured for illumination in a epi-illumination mode. The firstset of concentric rings 110(e)(1) are equally-spaced with a uniformradial spacing of Δr₁. The second set of concentric rings 110(e)(1) areequally-spaced with a uniform radial spacing of Δr₂. The first set ofconcentric rings 110(e)(1) are located at a distance, equal to theworking distance d_(o), above the sample 20.

In this illustrated example, the first set of concentric rings 110(e)(1)are centered around a central axis of the imaging lens 138 so that thethe first set does not have light elements 112(1) across the center ofthe imaging lens 138. The second set of first set of concentric rings110(e)(1) has light elements 112(2) configured to provide illuminationreflected by the beam splitter 139 through the imaging lens 138. Thesecond set of concentric rings 110(e)(2) comprises light elements 112(2)located at a plane that is at a combined optical path (a+b) of a focallength f from the imaging lens 138.

In FIG. 6, the variable-illumination Fourier ptychographic imagingdevice 100(e) is shown at a single illumination time and/or acquisitiontime. At this time, a single light element 112(1) from the from thefirst set of concentric rings 110(e)(1) is shown providing illumination114 at an incidence angle of θ_(B) with a wavevector of (kx,ky). Atother times, the other light elements 112(1) or 112(2) may be providingillumination. If one of the light elements 112(2) is illuminated,incident light is received by the beam splitter 139. Half the incidentlight received at the beam splitter 139 (and not absorbed) is reflectedto the imaging lens 138 which propates illumination to the sample 20.Since the beam splitter 139 passes half the incident illumination, incertain aspects, each of the light elements 112(2) of the second set ofconcentric rings 110(e)(2) has a light source with about two (2) times(2×) the intensity of the light source of each of the light elements112(1) of the first set of concentric rings 110(e)(1). In certain cases,the intensity from the light elements 112(2) may be adjusted to provideincident illumination at the sample 20 of about the same intensity asthe incident illumination provided by the light elements 112(1).

In an example operation of a system comprising the variable illuminatorof the variable-illumination Fourier ptychographic imaging device100(e), the light elements 112(1) and 112(2) of the variable illuminatorgenerate illumination directed to the sample at a plurality of Nincidence angles. Light reflected by the sample 20 is received at theimaging lens 138. The imaging lens 138 receives light within itsacceptance angle to filter the light. The imaging lens 138 propagatesincident light to the beam splitter 138. Half the incident light fromthe imaging lens 138 is transmitted through the beam splitter 138 andpropated to the radiation detector 140(e), which measures the intensitydistribution at different acquisition times to captures a plurality ofintensity images at different incidence angles.

FIGS. 7A and 7B depict illustrations of orthogonal views of componentsof a variable-illumination Fourier ptychographic imaging device 100(f),according to certain embodiments. FIG. 7A illustrates the illuminationscheme and FIG. 7B illustrates the collection scheme of thevariable-illumination Fourier ptychographic imaging device 100(f).Certain components of the variable-illumination Fourier ptychographicimaging device 100(f) are similar to components of othervariable-illumination Fourier ptychographic imaging devices of otherillustrations.

In FIGS. 7A and 7B, the variable-illumination Fourier ptychographicimaging device 100(f) comprises a variable illuminator 110(f) comprisestwelve (12) concentric rings 110(e)(1) of light elements 112, an opticalsystem, and a radiation detector 140(e) having a detector plane 142. Thevariable illuminator 110(f) comprises twelve (12) concentric rings110(e)(1) of light elements 112. Other numbers of concentric rings maybe used such as, for example, 1, 2, 3, 4, 5, 6, 7, 8, 9, . . . 13, 14,15, etc. The outermost concentric ring has a width w. The optical systemcomprises a objective 134 (e.g., microscope objective) with a focallength f, a tube lens 132, a secondary lens 138, and a beam splitter139. Although the objective 134 is illustrated here as a microscopeobjective, another objective may be used. A sample 20 is shown on aspecimen surface 126 as provided to the variable-illumination Fourierptychographic imaging device 100(f). The illustrated example shows aworking distance d_(o) between the objective 134 and the sample 20. Inthe illustrated example, a microscope objective may be used so that theconfiguration has a short working distance such as, for example, 2-3 cm.One operational range could be with a 0.08 NA 2× objective lens with a˜2 cm working distance. Another could be with a 20×0.5 NA objective lenswith a ˜2 mm working distance.

In the illustrated configuration, the entire variable illuminator 110(f)(e.g., LED array) is located behind the objective 134 (primary imagingoptics) and a secondary lens 130 is used to image the variableilluminator 110(f) to a back focal plane of the objective. In FIGS. 7Aand 7B, the optical path distance between the beam splitter 139 and thesecondary lens 138 is d_(i1), the optical path distance between thesecondary lens 138 is d₂, and the optical path distance between the beamsplitter 139 and the back focal plane of the objective 134 is d_(i2). InFIG. 7A, an image 136 of the variable illuminator 110(f) is shown at theback focal plane 135 an optical distance of a focal length F from theback of the objective 134. To assure that the variable illuminator110(f) image is formed on the back focal plane 135 of the objective 134,the components of the optical system are located so that the opticalpath distances follow this equation: 1/f=1/d₂+1/(d_(i1)+d_(i2)). FIG. 7Balso shows an optical distance of d_(s) from the tube lens 132 to theradiation detector 140(f) and an d_(t) between from the tube lens 132 tothe back of the objective 134. The illustrated example includes anobjective 134 that is a 2× microscope objective. In other examples,other objectives may be used. An example of values that can be used inthe illustrated configuration are w=10 cm, the d₂=20 cm, thed_(i1)+d₁₂=2 cm, and the f=1.9 cm. Other values can be used.

The beam-splitter 139 is configured to transmit half the illuminationincident at a 45 degree angle to the beam-splitter 139 and not absorbedby the beam-splitter 139. The remaining half of the incidentillumination (not absorbed) is reflected by the beam-splitter 139. Forexample, the the beam splitter 139 may be comprised of a sheet of glassor other substrate with a coating designed to control the lightaccordingly. As another example, a beam splitter may be a half-silveredmirror with a continuous thin coating of reflective material (e.g.,metal). Another example is a swiss cheese beam splitter which has adiscontinuous coating with holes to obtain the desired ratio ofreflection to transmission.

In FIG. 7A, the resolution variable-illumination Fourier ptychographicimaging device 100(f) is shown at a single illumination time. At thistime, a single light element 112 of the variable illuminator 110(f) isactivated to provide illumination at an incidence angle of θ_(B) with awavevector of (kx,ky). At other times, other light elements 112 may beproviding illumination at other incidencd angles. Each light element 112includes a light source that can provide illumination (e.g.,approximately plane wave illumination) at a particular incidence angleto the sample 20.

As shown in FIG. 7A, during an operation of a system comprising thevariable illuminator of the variable-illumination Fourier ptychographicimaging device 100(f), different light elements 112 of the variableilluminator 110(f) are illuminated at different times. The secondarylens 138 receives illumination from the illuminated light element(s) 112and propagates the illumination to the beam splitter 139. The beamsplitter 139 transmits half the incident light and reflects half theincident light. The objective 134 propagates incident light to thesample to illuminate it at a plurality of N incidence angles atdifferent times. As shown in FIG. 7B, during the operation of thesystem, light issuing from the sample 20 is received by the objective134 acting as the filtering optical element of the optical system. Theobjective 134 propagates light to the beam splitter 139, which transmitshalf the light, not absorbed, and reflects the remainder. The tube lens132 receives light passing through the beam splitter 139 and propageslight to the radiation detector 140(f). The radiation detector 140(w)measures the intensity distribution at different acquisition times tocapture a plurality of intensity images at different incidence angles.

II. Variable-illumination Fourier Ptychographic Imaging Methods

In certain aspects, a variable-illumination Fourier ptychographicimaging method comprises a measurement process, a recovery process, andan optional display process. During the measurement process, the sampleis illuminated from a plurality of N incidence angles (θx_(i,j),θy_(i,j)), i=1 to n, j=1 to m, (N=n×m) using a variable illuminator.During this process, the optical system filters the light issuing fromthe illuminated sample to propagate filtered light to the radiationdetector and the radiation detector receives the filtered light andacquires a plurality of M intensity images, k_(k,l), k=1 to o and j=1 top, where M=o×p. In certain cases, an intensity image is captured at eachincidence angle. In certain aspects, the variable illuminator may bedesigned to generate illumination at certain incidence angles thatgenerate intensity data that corresponds to regions that overlap in theFourier domain by a certain amount and also cover outer higher frequencyarea. During the recovery process, the M intensity images areiteratively combined in the Fourier domain to generate higher-resolutionimage data (intensity and/or phase). At each iteration, a filter isapplied in the Fourier domain for a particular plane wave incidenceangle, an inverse Fourier transform is applied to generate a lowerresolution image, the intensity of the lower resolution image isreplaced with an intensity measurement from the radiation detector, aFourier transform is applied, and the corresponding region in Fourierspace is updated. During the optional display process, an image (e.g.,higher-resolution image, acquired intensity image, etc.) and/or otheroutput may be provided on a display. Generally, these methods alternatebetween two working domains: the spatial (x-y) domain and the Fourier(kx-ky) domain, where k represents the wavenumber.

In certain aspects, variable-illumination Fourier ptychographic imagingmethods may comprise a phase retrieval technique that uses angulardiversity to recover complex sample images. The recovery processalternates enforcement of known image data acquired in the spatialdomain and a fixed constraint in the Fourier domain. This phaseretrieval recovery can be implemented using various methods such as, forexample, an alternating projections procedure, a convex reformulation ofthe problem, or any non-convex variant in-between. Instead of needing totranslate a sample laterally (i.e. applying translational diversity),variable-illumination Fourier ptychographic imaging systems use methodsthat vary the spectrum constraint in the Fourier domain to expand theFourier passband beyond that of a single captured image to recover ahigher-resolution sample image.

In some cases, variable-illumination Fourier ptychographic imagingmethods may also comprise an optional aberration correction process. Anexample of an aberration correction process is a re-focusing(propagating) process. Such a refocusing process may be useful where thesample was placed at a sample plane at z=z₀ where the in-focus plane ofthe optical element is located at position z=0. In other words, theimage captured of the sample is not the image at the sample plane, butis the sample profile propagated by a distance of −z₀ from the in-focusplane of the optical element. In these cases, the method may re-focusthe sample by propagating the image data by the z₀ distance back to thesample plane, without having to mechanically move the sample in thez-direction. The re-focusing (propagating) step(s) can be performed bymultiplying a phase factor in Fourier space.

With reference to certain illustrated examples, subscript “h” refers tohigher-resolution, subscript “l” refers to lower resolution intensity,subscript “f” refers to focused position, subscript “m” refers tomeasured, and subscript “s” refers to sampled.

FIG. 8 is a flowchart depicting steps of a variable-illumination Fourierptychographic imaging method, according to certain embodiments. Thismethod is performed by a variable-illumination Fourier ptychographicimaging system such as, for example, the system 10 described withreference to FIG. 1. The variable-illumination Fourier ptychographicimaging method comprises a measurement process (steps 1100, 1200, and1300), a recovery process (steps 1400 and 1500), and an optional displayprocess (step 1600).

At step 1100, a variable illuminator provides illumination to a samplefrom a plurality of N incidence angles (θx_(i,j), θy_(i,j)), i=1 to n,j=1 to m, at N sample times. In some cases, the variable illuminatorcontrols the illumination provided to the sample based on illuminationinstructions. The illumination instructions may define the order of theillumination angles and the associated illumination time. The wavevector in x and y directions can be denoted as wavevector kx_(i,j),ky_(i,j).

In certain aspects, the variable illuminator may provide illumination ofdifferent wavelengths at different sample times. For example, thevariable illuminator may provide RGB illumination of three wavelengthsλ₁, λ₂, and λ₃ corresponding to red, green, blue colors, respectively,at different sample times, for example, in a color imaging embodiment.

In some cases, the variable illuminator is configured to provide planewave illumination. Plane wave illumination with a wavevector, kx, ky, inthe spatial domain, is equivalent to shifting the center of the imagespectrum by (kx, ky) in the Fourier domain. In this respect, theintensity image data in the Fourier domain is shifted from normalincidence image data by (kx, ky), which corresponds to the incidenceangle (θx, θy) applied by the variable illuminator.

At step 1200, the optical system collects light issuing from the sampleand propagates it to the radiation detector. The optical systemcomprises a filtering optical element(s) that filters the light. Forexample, a filtering optical element may be an objective lens collectinglight issuing from an illuminated sample. In this case, the objectivelens filters the light issuing from the sample by only accepting lightincident at a range of angles within its numerical aperture (NA). InFourier space, the filtering function of a filtering optical elementsuch as an objective lens may be represented by a circular pupil withradius of NA×k₀, where k₀=2π/λ is the wave number in vacuum. That is,the variable-illumination Fourier ptychographic imaging method mayupdate in Fourier space circular regions defined by this filteringfunction and the different incidence angles. In certain cases, thefiltering optical element and its associated filtering function omitsdata outside the circular pupil region.

At step 1300, the radiation detector receives light propagated by theoptical system and captures a snapshot intensity distributionmeasurement at each of the M sample times, t_(k), k=1 to M, to acquire aplurality of M intensity images, I_(k,1, k=)1 to o and j=1 to p,associated with different incidence angles. Each intensity image sampledby the radiation detector is associated with a region in Fourier space.In many aspects, the variable illuminator is configured to provideillumination from incidence angles that will generate overlapping areasbetween neighboring (adjacent) regions (e.g., circular pupil regions) inFourier space. In one aspect, the variable illuminator is designed toprovide an overlapping area between neighboring regions of 2% to 99.5%of the area of one of the regions. In another aspect, the variableilluminator is designed to provide an overlapping area betweenneighboring regions of 65% to 75% of the area of one of the regions. Inone aspect, the variable illuminator is designed to provide anoverlapping area between neighboring regions of about 65% of the area ofone of the regions.

In steps 1400 and 1500, a higher-resolution image of the sample may begenerated from the M intensity distribution measurements acquired atstep 1300. The M intensity images, I_(k,l), k=1 to o and j=1 topcorrespond to different incidence angles indexed by illuminationwavevector kx_(i,j), ky_(i,j), i=1 to n, j=1 to m. At step 1400, ahigher-resolution image: √{square root over (I_(h))}e^(iφ) ^(h) isinitialized in the spatial domain, and a Fourier transform is applied tothe initial value to obtain an initialized Fourier transformed imageĨ_(h). The initialized higher-resolution solution may be an initialguess. This initial guess may be determined based on the assumption thatthe sample is located at the out-of-focus plane z=z₀. In some cases, theinitial guess may be determined as a random complex matrix (for bothintensity and phase). In other cases, the initial guess may bedetermined as an interpolation of the low-resolution intensitymeasurement with a random phase. An example of an initial guess is φ=0and I_(h) interpolated from any lower-resolution image of the samplearea. Another example of an initial guess is a constant value. TheFourier transform of the initial guess can be a broad spectrum in theFourier domain. At step 1500, the higher-resolution image of the sampleis constructed by iteratively combining low-resolution intensitymeasurements in Fourier space. In many cases, portions of step 1500 maybe implemented using a processor (e.g., processor 210 of the system 10).

At optional step 1600, the display may receive image data such as thehigher-resolution image data and/or other data from the processor, anddisplay the data on a display (e.g., display 230 in FIG. 1).

Aberration Correction

In certain aspects, the recovery process step 1500 may comprise anaberration correction process that introduces a phase map to thefiltering function to compensate for aberrations at the pupil planeduring the iterative image recovery process. FIG. 9 is a flowchartdepicting an example of sub-steps of step 1500 of thevariable-illumination Fourier ptychographic imaging method of FIG. 8that optionally comprises an aberration correction process, according tocertain aspects. In the illustrated flowchart, the optional aberrationcorrection process comprises incorporating compensation at the twomultiplication steps 1610 and 1645. Step 1610 models the connectionbetween the actual sample profile and the captured intensity data (withincludes aberrations) through multiplication with a pupil function:e^(i·φ(k) ^(x) ^(,k) ^(y) ⁾.

Step 1645 inverts such a connection to achieve an aberration-freereconstructed image. For example, aberration correction can correctsample defocus. In certain cases, sample defocus may be essentiallyequivalent to introducing the following defocus phase factor to thepupil plane (i.e., a defocus aberration):

e ^(i·φ(k) ^(x) ^(,k) ^(y) ⁾ =e ^(i)√{square root over (^((2π/λ)) ²^(−k) ^(x) ² ^(−k) ^(y) ² )}^(·z) ⁰ , k _(x) ² +k _(y) ²<(NA·2π/λ)²   (1)

where kx and ky are the wavenumbers at the pupil plane, z₀ is thedefocus distance, and NA is the numerical aperture of the filteringoptical element.

At step 1605, a processor performs filtering of the higher-resolutionimage √{square root over (I_(h))}e^(iφ) ^(h) in the Fourier domain togenerate a lower-resolution image √{square root over (I_(l))}e^(iφ) ^(l)for a particular plane wave incidence angle (θ_(x) ^(i), θ_(y) ^(i))with a wave vector (kx_(i,j), ky_(i,j)). The Fourier transform of thehigher-resolution image is Ĩ_(h) and the Fourier transform of thelower-resolution image for a particular plane wave incidence angle isĨ_(l). In the Fourier domain, the method filters a region from thespectrum Ĩ_(h), of the higher-resolution image √{square root over(I_(h))}e^(iφ) ^(h) . In cases with a filtering optical element in theform of an objective lens, this region is a circular pupil aperture witha radius of NA*k₀, where k₀ equals 2π/λ (the wave number in vacuum),given by the coherent transfer function of an objective lens. In Fourierspace, the location of the region (e.g., location of center of circularregion) corresponds to the corresponding incidence angle. For an obliqueplane wave incidence with a wave vector (kx_(i,j), ky_(i,j)), the regionis centered about a position (kx_(i,j), ky_(i,j)) in the Fourier domainof √{square root over (I_(h))}e^(iφ) ^(h) .

At optional step 1610, the processor may multiply by a phase factore^(i·φ(k) ^(x) ^(,k) ^(y) ⁾ in the Fourier domain as part of aberrationcompensation.

At step 1625, an inverse Fourier transform is taken to generate thelower resolution image region √{square root over (I_(lf))}e^(iφ) ^(l) .

At step 1630, the computed amplitude component √{square root over(I_(lf))} of the lower-resolution image region at the in-focus plane,√{square root over (I_(lf))}e^(iφ) ^(lf) , is replaced with thelow-resolution intensity measurement √{square root over (I_(lfm))}captured by the radiation detector. This forms an updated lowerresolution image: √{square root over (I_(lfm))}e^(iφ) ^(lf) . A Fouriertransform is then applied to the updated lower resolution image data.

At optional step 1645, an inverse phase factor e^(−i·φ(k) ^(x) ^(,k)^(y) ⁾ is applied in the Fourier domain.

At step 1650, the corresponding region of the higher-resolution solution√{square root over (I_(h))}e^(iφ) ^(h) in the Fourier domaincorresponding to incidence wave vector (kx, ky) is updated with theupdated lower resolution image data.

At step 1660, it is determined whether steps 1605 through 1650 have beencompleted for the different incidence angles associated with thecaptured images. If steps 1605 through 1650 have not been completed forthese different incidence angles, steps 1605 through 1650 are repeatedfor the next incidence angle. The next incident angle is typically thenext adjacent angle. In certain aspects, the neighboring (adjacent)regions are overlapping in Fourier space and are iteratively updated(e.g., by repeating steps 1605 through 1650 for each adjacent incidenceangle). At the overlapping area between adjacent regions, there is databased on multiple samplings over the same Fourier space. The incidenceangles of the illumination from the variable illuminator determine theoverlapping area between the regions. In one example, the overlappingarea between neighboring regions is in the range of about 2% to 99.5% ofthe area of one of the corresponding neighboring regions. In anotherexample, the overlapping area between neighboring regions is in therange of about 65% to 75% of the area of one of the correspondingneighboring regions. In another example, the overlapping area betweenneighboring regions is about 65% of the area of one of the correspondingneighboring regions. In another example, the overlapping area betweenneighboring regions is about 70% of the area of one of the correspondingneighboring regions. In another example, the overlapping area betweenneighboring regions is about 75% of the area of one of the correspondingneighboring regions. In certain embodiments, each overlapping region hasthe same area.

At step 1670, it is determined whether a higher-resolution image datahas converged. For example, a processor may determine whether thehigher-resolution image data may have converged to be self-consistent.In one case, a processor compares the previous higher-resolution imagedata of the previous iteration or initial guess to the presenthigher-resolution data, and if the difference is less than a certainvalue, the image data may have converged to be self-consistent. If it isdetermined that the image data has not converged, then steps 1605through 1670 are repeated. In one case, steps 1605 through 1670 arerepeated once. In other cases, steps 1605 through 1670 are repeatedtwice or more.

If the image data has converged, the converged image data in Fourierspace is transformed using an inverse Fourier transform to the spatialdomain to recover a higher-resolution image √{square root over(I_(h))}e^(iφ) ^(h) . If it is determines that the solution hasconverged at step 1670, then the method may proceed to optional step1600 or the method may end.

FIG. 10A is a flowchart depicting an example of sub-steps of step 1500shown in FIG. 8, according to an embodiment. These sub-steps comprise anoptional aberration correction process that corrects for defocus. InFIG. 10, step 1500 comprises step 1510, step 1530, step 1550, step 1560,step 1570, step 1580, and step 1590. In aspects that include aberrationcorrection, step 1500 may further incorporate compensation at the twomultiplication optional steps 1520 and 1540. For example, optional steps1520 and 1540 can be used to focus an out-of focus sample that isout-of-focus by the amount of z₀.

At step 1510, a processor performs filtering of the higher-resolutionimage √{square root over (I_(h))}e^(iφ) ^(h) in the Fourier domain togenerate a lower-resolution image √{square root over (I_(l))}e^(iφ) ^(l)for a particular plane wave incidence angle (θx_(i,j), θy_(i,k)) with awave vector (kx_(i,j), ky_(i,j)). The Fourier transform of thehigher-resolution image is Ĩ_(h) and the Fourier transform of thelower-resolution image for a particular plane wave incidence angle isĨ_(l). In the Fourier domain, the method filters a region from thespectrum Ĩ_(h) of the higher-resolution image √{square root over(I_(h))}e^(iφ) ^(h) . In cases with a filtering optical element in theform of an objective lens, this region is a circular pupil aperture witha radius of NA*k₀, where k₀ equals 2π/λ (the wave number in vacuum),given by the coherent transfer function of an objective lens. In Fourierspace, the location of the region (e.g., location of center of circularregion) corresponds to the corresponding incidence angle. For an obliqueplane wave incidence with a wave vector (kx_(i,j), ky_(i,j)), the regionis centered about a position (kx_(i,j), ky_(i,j)) in the Fourier domainof √{square root over (I_(h))}e^(iφ) ^(h) .

At optional step 1520, the low-resolution image, √{square root over(I_(l))}e^(iφ) ^(l) is propagated in the Fourier domain to an in-focusplane at z=0 of the optical system to determine the lower-resolutionimage at the focused position: √{square root over (I_(lf))}e^(iφ) ^(lf). In one case, optional step 1520 can be performed by Fouriertransforming the low-resolution image √{square root over (I_(l))}e^(iφ)^(l) , multiplying by a phase factor in the Fourier domain, and inverseFourier transforming to obtain √{square root over (I_(lf))}e^(iφ) ^(lf). In another case, optional step 1520 can be performed by themathematically equivalent operation of convolving the low-resolutionimage √{square root over (I_(l))}e^(iφ) ^(l) with thepoint-spread-function for the defocus. In another case, optional step1520 can be performed as an optional sub-step of step 1510 bymultiplying Ĩ_(l) by a phase factor in the Fourier domain beforeperforming the inverse Fourier transform to produce √{square root over(I_(lf))}e^(iφ) ^(lf) . In certain instances, optional step 1520 neednot be included if the sample is located at the in-focus plane (z=0) ofthe filtering optical element.

At step 1530, the computed amplitude component √{square root over(I_(lf))} of the lower-resolution image at the in-focus plane, √{squareroot over (I_(lf))}e^(iφ) ^(lf) , is replaced with the square root ofthe low-resolution intensity measurement √{square root over (I_(lfm))}measured by the radiation detector. This forms an updated low resolutiontarget: √{square root over (I_(lfm))}e^(iφ) ^(lf) .

At optional step 1540, the updated low-resolution image √{square rootover (I_(lfm))}e^(iφ) ^(lf) may be back-propagated to the sample plane(z=z₀) to determine √{square root over (I_(ls))}e^(iφ) ^(ls) . Incertain instances, optional step 1540 need not be included if the sampleis located at the in-focus plane of the filtering optical element, thatis, where z₀=0. In one case, step 1540 can be performed by taking theFourier transform of the updated low-resolution image √{square root over(I_(lfm))}e^(iφ) ^(lf) and multiplying in the Fourier space by a phasefactor, and then inverse Fourier transforming it. In another case, step1540 can be performed by convolving the updated low-resolution image√{square root over (I_(lfm))}e^(iφ) ^(lf) with the point-spread-functionof the defocus. In another case, step 1540 can be performed as asub-step of step 1550 by multiplying by a phase factor after performingthe Fourier transform onto the updated target image.

At step 1550, a Fourier transform is applied to the updated target imagepropagated to the sample plane: √{square root over (I_(ls))}e^(iφ) ^(ls), and this data is updated in the corresponding region ofhigher-resolution solution √{square root over (I_(h))}e^(i®) ^(h) in theFourier space corresponding to the corresponding to the incidence wavevector kx_(i,j), ky_(i,j).

At step 1560, it is determined whether steps 1510 through 1560 have beencompleted for the different incidence angles associated with thecaptured images. If steps 1605 through 1650 have not been completed forthese different incidence angles, steps 1510 through 1560 are repeatedfor the next incidence angle. The next incident angle is typically thenext adjacent angle. In certain aspects, the neighboring (adjacent)regions are overlapping in Fourier space and are iteratively updated(e.g., by repeating steps 1510 through 1560 for each adjacent incidenceangle). At the overlapping area between adjacent regions, there is databased on multiple samplings over the same Fourier space. The incidenceangles of the illumination from the variable illuminator determine theoverlapping area between the regions. In one example, the overlappingarea between neighboring regions is in the range of about 2% to 99.5% ofthe area of one of the corresponding neighboring regions. In anotherexample, the overlapping area between neighboring regions is in therange of about 65% to 75% of the area of one of the correspondingneighboring regions. In another example, the overlapping area betweenneighboring regions is about 65% of the area of one of the correspondingneighboring regions. In another example, the overlapping area betweenneighboring regions is about 70% of the area of one of the correspondingneighboring regions. In another example, the overlapping area betweenneighboring regions is about 75% of the area of one of the correspondingneighboring regions. In certain embodiments, each overlapping region hasthe same area.

At step 1570, it is determined whether a higher-resolution image datahas converged. For example, a processor may determine whether thehigher-resolution image data may have converged to be self-consistent.In one case, a processor compares the previous higher-resolution imagedata of the previous iteration or initial guess to the presenthigher-resolution data, and if the difference is less than a certainvalue, the image data may have converged to be self-consistent. If it isdetermined that the image data has not converged, then steps 1510through 1560 are repeated. In one case, steps 1510 through 1560 arerepeated once. In other cases, steps 1510 through 1560 are repeatedtwice or more. If the image data has converged, the converged image datain Fourier space is transformed using an inverse Fourier transform tothe spatial domain to recover a higher-resolution image √{square rootover (I_(h))}e^(iφ) ^(h) . If it is determines that the solution hasconverged at step 1570, then the method may proceed to optional step1600 or the method may end.

In certain aspects, the variable-illumination Fourier ptychographicimaging method described with reference to FIG. 8 can include anoptional aberration correction process described with reference toeither FIG. 9 or FIG. 10A. In one aspect, the variable-illuminationFourier ptychographic imaging method includes the optional aberrationcorrection process for refocusing described in optional steps 1520 and1540 of FIG. 1 to refocus. The refocusing feature of optional steps 1520and 1540 propagates the image from the in-focus plane z=0 to the sampleplane at z=z₀. Refocusing may be needed when the sample is located atthe sample plane at z=z₀, while the in-focus plane of the filteringoptical element (e.g., objective lens) is located at position z=0. Inother words, refocusing may be needed when the sample is out-of-focus bythe amount of z₀.

FIGS. 10B and 10C are schematic illustrations depicting components of avariable-illumination Fourier ptychographic imaging device 100(a) intrans-illumination mode, according to an embodiment. The Fourierptychographic imaging device 100(a) comprises a variable illuminator110(b) is in the form of a two-dimensional matrix of light elements(e.g. an LED matrix). In FIGS. 10B and 10C, a single light element 112of the variable illuminator 110(b) at X_(i,j) (x′,y′) is shown asilluminated at the sample time illustrated. The Fourier ptychographicimaging device 100(a) further comprises an optical system 130.

In FIG. 10C, the sample 20 is depicted as out-of-focus by an amount of−z₀, and optional steps 1520 and 1540 (depicted here as arrows) can beused to digitally refocus the sample 20 to the in-focus plane 122 asdepicted by the dottily line to the in-focus plane 122. In FIG. 6D, thesample 20 is located at in-focus plane 122. In this case, optional steps1520 and 1540 may not be needed.

FIG. 10D is an illustration of steps of the variable-illuminationFourier ptychographic imaging method described with reference to FIGS. 8and 10A, according to an embodiment. The left-hand-side image in FIG. 6Eincludes two circular regions 22(a) and 22(b) in Fourier space used togenerate the higher-resolution image region. The circular regions 22(a)and 22(b) may be defined NA of the filtering optical element based onapproximating as circular pupil function with a radius of NA*k₀, wherek₀ equals 2π/λ, (the wave number in vacuum). For example, each circularregion 22(a) and 22(b) may be defined by the optical transfer functionof a 2× objective lens 0.08 NA. In FIG. 10D, region 22(a) is of acircular low-pass filter shape associated with a plane wave incidenceangle: θ_(x)=0; θ_(y)=0, i=1 and Region 22(b) is of a circular low-passfilter shape associated with a plane wave incidence angle: θ_(x)=−21°;θ_(y)=22°. To perform filtering at each incidence angle, data outsidethe circular region in the Fourier domain is omitted, which results in alow-resolution data. The low-resolution image resulting from filteringbased on plane wave incidence angle of θ_(x)=−21; θ_(y)=22° is shown atthe top right-hand-side of FIG. 10D. The low-resolution image resultingfrom filtering based on plane wave incidence angle of θ_(x)=−21°;θ_(y)=22° is shown at the bottom right-hand-side of FIG. 10D. The wavevectors of the incidence angles in the x-direction and y-direction aredenoted as kx and ky respectively.

When implementing the updating step 1550 of FIG. 10A or the updatingstep 1650 of FIG. 9, the method updates the data within the region 22(a)of the higher-resolution reconstruction 22(c) corresponding to thenormal incidence θ_(x)=0, θ_(y)=0. The method also updates the datawithin the region 22(b) of the higher-resolution reconstructioncorresponding to the n^(th) incidence angle θ_(x)=−21°; θ_(y)=22°. Theregions are updated with low-resolution image measurement data.

Tile Imaging

In certain aspects, a variable-illumination Fourier ptychographicimaging method may comprise tile imaging to divide the capturedintensity images into a plurality of tile images, independently acquirea higher-resolution image for each of the tiles, and then combine thehigher-resolution tile images to generate a full field-of-viewhigher-resolution image. In some cases, the higher-resolution tileimages may be combined with an image blending process. An example of animage blending process is alpha blending which can be found in PCTpublication WO1999053469, entitled “A system and method for performingblending using an over sampled buffer,” filed on Apr. 7, 1999, which ishereby incorporated by reference in its entirety. Sincehigher-resolution images of the tiles may be acquired independently,this method may be well suited for parallel computing, which may reducecomputational time, and may also reduce memory requirements. Moreover,the light from each light element may be accurately treated as a planewave for each tile. The incident wavevector for each tile can beexpressed as:

$\begin{matrix}{\left( {k_{x}^{i},k_{y}^{i}} \right) = {\frac{2\pi}{\lambda}\left( {\frac{\left( {x_{c} - x_{i}} \right)}{\sqrt{\left( {x_{c} - x_{i}} \right)^{2} + \left( {y_{c} - y_{i}} \right)^{2} + h^{2}}},\frac{\left( {y_{c} - y_{i}} \right)}{\sqrt{\left( {x_{c} - x_{i}} \right)^{2} + \left( {y_{c} - y_{i}} \right)^{2} + h^{2}}}} \right)}} & \left( {{Eqn}.\mspace{14mu} 2} \right)\end{matrix}$

where (x_(c),y_(c)) is the central position of each tile of the fullfield-of-view low-resolution image, (x_(i),y_(i)) is the position of thei^(th) light element, and h is the distance between the variableilluminator and the sample. Furthermore, this method can assign aspecific aberration-correcting pupil function to each tile in somecases.

FIG. 11 is a flowchart depicting a variable-illumination Fourierptychographic imaging method which includes tile imaging, according toan embodiment. This method can be performed by variable-illuminationFourier ptychographic imaging system such as the system 10 illustratedin FIG. 1. To take advantage of parallel processing capabilities, theprocessor of the system should be configured with parallel processingcapabilities such as, for example, the GPU unit or a processor havingmultiple cores (i.e. independent central processing units).

In FIG. 11, the variable-illumination Fourier ptychographic imagingmethod comprises a measurement process (steps 1100, 1200, and 1300), arecovery process (steps 1350, 2400 (i-M), 2500(i-M), 2590), and anoptional display process (step 1600). The measurements process (steps1100, 1200, and 1300) and optional display process (step 1600) aredescribed with reference to FIG. 8.

At step 1350, the processor divides the full field-of-view into aplurality of tiles such as, for example, a two-dimensional matrix oftiles. The dimensions of a two-dimensional square matrix of tiles may bein powers of two such as, for example, a 256 by 256 matrix, a 64×64matrix, etc. In one example, the processor may divide up a full field ofview of 5,280×4,380 pixels into tiles having an area of 150×150 pixels.

Next, the processor initializes the higher-resolution image: √{squareroot over (I_(h))}e^(iφ) ^(h) in the spatial domain for each tile (1 toT) independently using parallel computing (step 2400(1) . . . step2400(T)). A Fourier transform is applied to the initial guess. In somecases, the initial guess may be determined as a random complex matrix(for both intensity and phase). In other cases, the initial guess may bedetermined as an interpolation of the low-resolution intensitymeasurement with a random phase. An example of an initial guess is φ=0and I_(k,l) of any low-resolution image of the sample area. Anotherexample of an initial guess is a constant value. The Fourier transformof the initial guess can be a broad spectrum in the Fourier domain.

At step 2500(1) . . . step 2500(T), the processor reconstructs ahigher-resolution image of each tile (1 to T) independently usingparallel computing. The processor reconstructs the higher-resolutionimage of each tile by iteratively combining low-resolution intensityimages in Fourier space as described with reference to steps 1510, 1530,1550, 1560, and 1570 shown in FIG. 6B, and described herein. Steps 1520and 1540 may be included if the sample is out of focus.

At step 2590, the processor combines the higher-resolution tile imagesinto a full field-of view higher-resolution image. In some cases,combining tile images comprises an imaging-blending process such as, forexample, alpha blending.

At optional step 2600, the image data of the recovered higher-resolutiontwo-dimensional image of the sample area is displayed on a display(e.g., display 230). In one aspect, the method with tile imaging mayfurther comprise a procedure that accounts for differences in incidentangles between different tiles based on the distance between the tilesand each light element.

Refocusing and Auto-Focusing

Conventional high NA microscopes and other imaging devices typicallyhave a limited depth of field. For example, the depth-of-field of aconventional microscope with a 20× objective lens with 0.4 NA is about 5μm. With a conventional microscope, resolution degrades as the samplemoves away from the in-focus plane due to its limited depth-of-field. Toimprove resolution using a conventional microscope, the operatortypically moves the stage to mechanically bring the sample back intofocus. In this regard, a precise mechanical stage is needed to bring asample into the in-focus position with sub-micron accuracy.

In certain aspects, a variable-illumination Fourier ptychographicimaging system can refocus the sample without mechanically moving thesample. For example, the variable-illumination Fourier ptychographicimaging method may comprise steps that refocus an out-of-focus sampleduring the recovery process. With this refocusing procedure, thevariable-illumination Fourier ptychographic imaging system can expandits depth-of focus beyond the physical limitations of its filteringoptical element. In certain cases, a variable-illumination Fourierptychographic imaging system may be able auto-focus the sample.

During operation of a variable-illumination Fourier ptychographicimaging system, the z-position of the sample plane may not be known apriori. In certain aspects, a variable-illumination Fourierptychographic imaging method may include one or more auto-focusing stepsthat determines the z-position of the sample plane and uses thisz-position to digitally refocus the sample. For example, the avariable-illumination Fourier ptychographic imaging method describedwith respect to FIG. 10A may further comprise a step during or beforestep 1520 that computes the z-position of the sample plane. Thevariable-illumination Fourier ptychographic imaging system may theperform autofocusing by using the processor to perform steps 1520 and1540 in FIG. 10A using the computed z-position of the sample. To computethe z-position of the sample plane, the method may determine anauto-focusing index parameter. The auto-focusing index is defined by thefollowing equation:

Auto-focusing index: 1/Σabs(√{square root over (I _(lf))}−√{square rootover (I_(lfm))})   (4)

Where: √{square root over (I_(lf))} is the amplitude image from thelow-pass filtering, and is the actual low-resolution measurement

The summation in Eqn. 4 is for all oblique incidence angles. After thevariable-illumination Fourier ptychographic imaging method computes theestimated z-position of the sample plane, the variable-illuminationFourier ptychographic imaging method can digitally refocus to theestimated z-position. In some cases, the higher-resolution imagesolution has been found to converge better when using an accuratez-position.

III. Subsystems

FIG. 12 is a block diagram of subsystems that may be present in certainvariable-illumination Fourier ptychographic imaging system describedherein. For example, a variable-illumination Fourier ptychographicimaging system may include a processor. The processor may be a componentof the variable-illumination Fourier ptychographic imaging system insome cases. The processor may be a component of the radiation detectorin some cases.

The various components previously described in the Figures may operateusing one or more of the subsystems to facilitate the functionsdescribed herein. Any of the components in the Figures may use anysuitable number of subsystems to facilitate the functions describedherein. Examples of such subsystems and/or components are shown in aFIG. 12. The subsystems shown in FIG. 12 are interconnected via a systembus 2425. Additional subsystems such as a printer 2430, keyboard 2432,fixed disk 2434 (or other memory comprising computer readable media),display 230, which is coupled to display adapter 2438, and others areshown. Peripherals and input/output (I/O) devices, which couple to I/Ocontroller 2440, can be connected by any number of means known in theart, such as serial port 2442. For example, serial port 2442 or externalinterface 2444 can be used to connect the computing device 200 to a widearea network such as the Internet, a mouse input device, or a scanner.The interconnection via system bus 2425 allows the processor tocommunicate with each subsystem and to control the execution ofinstructions from system memory 2446 or the fixed disk 2434, as well asthe exchange of information between subsystems. The system memory 2446and/or the fixed disk 2434 may embody the CRM 220 in some cases. Any ofthese elements may be present in the previously described features.

In some embodiments, an output device such as the printer 2430 ordisplay 230 of the aperture scanning Fourier ptychographic system canoutput various forms of data. For example, the aperture scanning Fourierptychographic system can output 2D color/monochromatic images (intensityand/or phase), data associated with these images, or other dataassociated with analyses performed by the aperture scanning Fourierptychographic system.

Modifications, additions, or omissions may be made to any of theabove-described embodiments without departing from the scope of thedisclosure. Any of the embodiments described above may include more,fewer, or other features without departing from the scope of thedisclosure. Additionally, the steps of the described features may beperformed in any suitable order without departing from the scope of thedisclosure.

It should be understood that certain features of embodiments of thedisclosure described above can be implemented in the form of controllogic using computer software in a modular or integrated manner. Basedon the disclosure and teachings provided herein, a person of ordinaryskill in the art will know and appreciate other ways and/or methods toimplement certain features using hardware and a combination of hardwareand software.

Any of the software components or functions described in thisapplication, may be implemented as software code to be executed by aprocessor using any suitable computer language such as, for example,Java, C++ or Perl using, for example, conventional or object-orientedtechniques. The software code may be stored as a series of instructions,or commands on a CRM, such as a random access memory (RAM), a read onlymemory (ROM), a magnetic medium such as a hard-drive or a floppy disk,or an optical medium such as a CD-ROM. Any such CRM may reside on orwithin a single computational apparatus, and may be present on or withindifferent computational apparatuses within a system or network.

Although the foregoing disclosed embodiments have been described in somedetail to facilitate understanding, the described embodiments are to beconsidered illustrative and not limiting. It will be apparent to one ofordinary skill in the art that certain changes and modifications can bepracticed within the scope of the appended claims.

One or more features from any embodiment may be combined with one ormore features of any other embodiment without departing from the scopeof the disclosure. Further, modifications, additions, or omissions maybe made to any embodiment without departing from the scope of thedisclosure. The components of any embodiment may be integrated orseparated according to particular needs without departing from the scopeof the disclosure.

1. An ultra-high NA Fourier ptychographic imaging system, comprising: avariable illuminator configured to illuminate a sample at a plurality ofincidence angles at different times; an optical system comprising a lenswith a high NA, the lens configured to filter light issuing from thesample, wherein the plurality of incidence angles and the high NAcorrespond to a plurality of overlapping regions in the Fourier domainthat cover an expanded NA of greater than 1.0; and a radiation detectorconfigured to acquire a plurality of intensity images, each intensityimage corresponding to a different incidence angle of the plurality ofincidence angles.
 2. The ultra-high NA Fourier ptychographic imagingsystem of claim 1, further comprising a processor configured to generatean image with a higher resolution than a resolution of the intensityimages by iteratively updating the overlapping regions in the Fourierdomain with intensity image measurements.
 3. The ultra-high NA Fourierptychographic imaging system of claim 1, wherein the lens is configuredto filter light from the sample by passing light received within itsacceptance angle. 4-5. (canceled)
 6. The ultra-high NA Fourierptychographic imaging system of claim 1, wherein the variableilluminator comprises one or more circular rings of light elements. 7.The ultra-high NA Fourier ptychographic imaging system of claim 1,wherein the variable illuminator comprises a plurality of concentricrings of equally-spaced light elements.
 8. The ultra-high NA Fourierptychographic imaging system of claim 7, wherein each outer ring has alarger number of light elements than an adjacent smaller diameter ring.9. The ultra-high NA Fourier ptychographic imaging system of claim 7,wherein each concentric ring has at least 6 light elements.
 10. Theultra-high NA Fourier ptychographic imaging system of claim 7, whereineach concentric ring light elements separated by at least about 30degrees. 11-12. (canceled)
 13. The ultra-high NA Fourier ptychographicimaging system of claim 1, wherein the optical system comprises acollection optical element configured to receive light reflected fromthe sample, and wherein the variable illuminator and the collectionoptical element are located to the same side of the sample in anepi-illumination mode.
 14. The ultra-high NA Fourier ptychographicimaging system of claim 1, wherein the lens is configured to receivelight reflected from the sample, and wherein the variable illuminatorand the lens optical element are located to the same side of the samplein an epi-illumination mode.
 15. The ultra-high NA Fourier ptychographicimaging system of claim 1, wherein adjacent overlapping regions of theplurality of overlapping regions have an overlapping area of at leastabout 20% to 90% of the area of one of the overlapping regions.
 16. Theultra-high NA Fourier ptychographic imaging system of claim 1, whereinadjacent overlapping regions of the plurality of overlapping regionshave an overlapping area of at least about 70% of the area of one of theoverlapping regions.
 17. The ultra-high NA Fourier ptychographic imagingsystem of claim 1, wherein adjacent overlapping regions of the pluralityof overlapping regions have an overlapping area of at least about 75% ofthe area of one of the overlapping regions.
 18. The ultra-high NAFourier ptychographic imaging system of claim 1, wherein adjacentoverlapping regions of the plurality of overlapping regions have anoverlapping area of at least about 2% and 99.5% of the area of one ofthe overlapping regions.
 19. A reflective-mode Fourier ptychographicimaging system, comprising: a variable illuminator configured toilluminate a sample at a plurality of incidence angles at differenttimes in an epi-illumination mode; an optical system comprising afiltering optical element having a filtering function, the opticalsystem configured to receive light reflected from the sample and filterthe light reflected from the sample using the filtering optical element,wherein the plurality of incidence angles and the filtering functioncorrespond to overlapping regions in the Fourier domain; and a radiationdetector configured to acquire a plurality of intensity images, eachintensity image corresponding to a different incidence angle of theplurality of incidence angles.
 20. The reflective-mode Fourierptychographic imaging system of claim 19, further comprising a processorconfigured to generate an image with a higher resolution than aresolution of the intensity images by iteratively updating theoverlapping regions in the Fourier domain with intensity imagemeasurements.
 21. The reflective-mode Fourier ptychographic imagingsystem of claim 19, wherein the filtering optical element is a lensconfigured to filter light by passing light received within itsacceptance angle.
 22. The reflective-mode Fourier ptychographic imagingsystem of claim 19, wherein the variable illuminator comprises a firstset of circular rings of light elements centered about a central axis ofthe filtering optical element.
 23. (canceled)
 24. The reflective-modeFourier ptychographic imaging system of claim 19, wherein the opticalsystem further comprises a beam splitter placed at a 45 degree anglebehind the filtering optical element; wherein the filtering opticalelement is configured to filter light issued from the sample; andwherein the beam splitter is configured to receive light filtered by thefiltering optical element and passes half the filtered light to theradiation detector.
 25. The reflective-mode Fourier ptychographicimaging system of claim 19, wherein the optical system further comprisesa secondary lens; and wherein the secondary lens is configured toreceive illumination at a plurality of incidence angles from thevariable illuminator and passes the illumination to the beam splitter,wherein the beam splitter is configured to pass half the illumination tothe sample through the filtering optical element.